Pseudo-Rehearsal

The fundamental mechanism of pseudo-rehearsal is the use of a generative model—such as a Generative Adversarial Network (GAN) or Variational Autoencoder (VAE)—to produce pseudo-examples that approximate the data distribution of previously learned tasks. This eliminates the need to store raw data, addressing privacy and storage constraints. The model is trained to generate these examples on-demand, creating a replay buffer of synthetic data for rehearsal.

• Key Component: A trained generator network that captures the manifold of past tasks.
• Primary Advantage: Provides a privacy-preserving, storage-efficient alternative to storing real exemplars.
• Critical Challenge: Requires the generative model itself not to suffer from catastrophic forgetting.

Pseudo-rehearsal relies heavily on knowledge distillation to transfer and consolidate knowledge. The current model's predictions on the generated pseudo-data serve as soft targets. When learning a new task, the model is trained on a mixture of new real data and old synthetic data, with a distillation loss (e.g., Kullback-Leibler Divergence) applied to the synthetic portion to preserve old responses.

• Mechanism: The model distills its own prior knowledge from its parameters into its future self via synthetic data.
• Loss Function: Combines a standard cross-entropy loss for the new task with a KL-divergence loss for the pseudo-rehearsal data.
• Relation to LwF: This is conceptually similar to Learning without Forgetting (LwF), but uses generated data instead of raw new-task data for distillation.

Pseudo-rehearsal typically operates under a two-model paradigm: a main task model (the classifier or predictor) and a separate generative model. These are often trained in alternating or coupled phases.

1. Phase 1 - Task Learning: The main model learns a new task with real data.
2. Phase 2 - Generative Replay: The generative model is updated to produce data for all tasks seen so far (including the new one).
3. Phase 3 - Consolidation: The main model is rehearsed on pseudo-data from all previous tasks generated in Phase 2.

This decoupling allows specialization but introduces complexity in coordinating the learning dynamics of both models.

A defining characteristic of pseudo-rehearsal is its suitability for task-agnostic continual learning. Since the generative model produces data from the aggregate distribution of all past experiences, the main model rehearses on a blended stream of pseudo-examples without explicit task labels or boundaries. This forces the model to develop a unified, generalizable representation that is robust to interference.

• Inference: The model does not require a task identifier at test time.
• Training: The pseudo-rehearsal batch is a random mix of synthetically generated classes from all historical tasks.
• Contrast with GEM/iCaRL: Unlike Gradient Episodic Memory (GEM) or iCaRL, it does not rely on task-specific constraints or exemplar sets.

Pseudo-rehearsal directly addresses the core plasticity-stability trade-off in continual learning.

• Stability is enforced via the distillation loss on pseudo-data, penalizing deviation from previous predictions.
• Plasticity is maintained by allocating full cross-entropy loss learning capacity to the new, real data.

The ratio of real to pseudo-data in each training batch, along with the weighting of the distillation loss, becomes a critical hyperparameter. An imbalance can lead to overfitting to new tasks (instability) or stagnation (lack of plasticity).

Despite its strengths, pseudo-rehearsal has distinct limitations:

• Generative Modeling Bottleneck: Performance is capped by the quality and diversity (mode coverage) of the synthetic data. A poor generator leads to catastrophic forgetting.
• Cumulative Error: Imperfections in generated data can compound over sequential tasks, leading to representation drift.
• Computational Overhead: Training and sampling from a generative model for every rehearsal step adds significant compute cost compared to simple experience replay with a stored buffer.
• Double Forgetting Risk: Both the main model and the generative model must be protected from forgetting, creating a more complex stability problem.

Generative replay is a memory consolidation method where a generative model (e.g., a GAN or VAE) is trained to produce synthetic data representative of past tasks. This pseudo-data is then interleaved with new task data during training. It is the conceptual parent of pseudo-rehearsal, with the key distinction being the use of a separate, dedicated generative model versus the primary model generating its own pseudo-data.

• Core Mechanism: A separate generative model learns the data distribution of previous tasks.
• Training Loop: For a new task, the generative model creates samples of old data, which are mixed with new real data for training.
• Relation to Pseudo-Rehearsal: Pseudo-rehearsal can be seen as a form of generative replay where the primary model acts as its own generator, eliminating the need for a separate generative network.

Experience replay is a fundamental continual learning mechanism where a subset of real past training examples is stored in a memory buffer (or replay buffer). During learning on a new task, these stored exemplars are sampled and 'replayed' alongside the new data. This provides direct exposure to the original data distribution of old tasks.

• Core Mechanism: Storage and retrieval of a limited set of real past data samples.
• Memory Buffer: A fixed-size reservoir of actual training examples from previous tasks.
• Contrast with Pseudo-Rehearsal: Pseudo-rehearsal generates synthetic examples, avoiding the storage of real data (beneficial for privacy or memory constraints) but potentially suffering from generation quality issues if the model's representation of past tasks degrades.

Learning without Forgetting (LwF) is a seminal distillation-based continual learning algorithm. When learning a new task, LwF uses the model's own outputs (logits) on the new task data—as they were before starting the new training—as soft targets for a knowledge distillation loss. This penalizes the model for changing its predictions on the new data in a way that would disrupt its knowledge of previous tasks.

• Core Mechanism: Self-distillation using the model's pre-update outputs as a stability anchor.
• No Exemplar Storage: Does not require storing any past data or generating pseudo-data.
• Relation to Pseudo-Rehearsal: Both are rehearsal-free methods. LwF uses self-distillation on new data, while pseudo-rehearsal uses self-generation of old data. They represent different approaches to constraining weight updates to preserve past knowledge.

Elastic Weight Consolidation (EWC) is a regularization-based continual learning method. It estimates the importance (Fisher information) of each neural network parameter to previously learned tasks. During training on a new task, EWC adds a quadratic penalty term to the loss function that heavily penalizes changes to parameters deemed important for old tasks, effectively 'consolidating' them.

• Core Mechanism: Computes a parameter importance matrix and uses it for regularization.
• Theoretical Basis: Grounded in Bayesian inference, approximating the posterior distribution of parameters given past data.
• Contrast with Pseudo-Rehearsal: EWC is a pure regularization approach that does not involve data replay (real or synthetic). Pseudo-rehearsal is a data-centric approach that uses generated data to create a rehearsal-like effect. They address the plasticity-stability trade-off from different angles.

Catastrophic forgetting (or catastrophic interference) is the primary problem that pseudo-rehearsal aims to solve. It is the tendency of an artificial neural network to abruptly and drastically lose performance on previously learned tasks or data distributions when it is trained on new, different tasks. This occurs because gradient-based optimization for the new task causes unconstrained overwriting of the weights that encoded the old knowledge.

• Root Cause: Non-stationary data distributions and overlapping parameter representations in neural networks.
• Manifestation: High accuracy on Task A drops to near-random levels after training on Task B.
• Pseudo-Rehearsal's Role: By generating and training on pseudo-data from past tasks, pseudo-rehearsal provides a regularizing signal that anchors the network's parameters, mitigating unconstrained drift and preserving old knowledge.

The plasticity-stability trade-off is the fundamental, competing-objective challenge at the heart of all continual learning, including pseudo-rehearsal. A model must maintain stability (retaining knowledge from past tasks) while exhibiting plasticity (efficiently learning new tasks). Over-emphasizing stability leads to intransigence (inability to learn new things), while over-emphasizing plasticity leads to catastrophic forgetting.

• Stability: The resistance of a model to changing its behavior on previously learned tasks.
• Plasticity: The capacity of a model to adapt its parameters to incorporate new information.
• Pseudo-Rehearsal's Balance: It explicitly manages this trade-off by allocating training cycles between new real data (promoting plasticity) and generated pseudo-data from old tasks (promoting stability). The quality and proportion of pseudo-data are critical hyperparameters for this balance.