Catastrophic Forgetting

The fundamental cause is the overwriting of shared synaptic weights in a neural network. When a model with a fixed architecture learns a new task (Task B), the gradient descent optimization process adjusts the same set of weights that were previously tuned for an old task (Task A). This adjustment moves the weights to a configuration optimal for Task B, often erasing the weight configuration that encoded knowledge of Task A. The problem is most severe in dense, fully-connected networks where weights are highly interdependent.

Catastrophic Forgetting occurs under non-stationary data distributions, a core challenge in Continual Learning. When the model is trained sequentially on tasks A, B, C..., the input-output mapping it must learn changes abruptly. The model's objective shifts from minimizing loss on Task A to minimizing loss on Task B. Without explicit constraints, the optimizer has no incentive to preserve performance on Task A, leading to unconstrained interference where new learning directly disrupts old representations. This is distinct from standard training where all data is assumed to be independent and identically distributed (IID).

A primary mechanism is the absence of interleaved training data. In biological systems, memory consolidation involves reactivation. In artificial neural networks, the equivalent is rehearsal or experience replay. When a model learns Task B without ever revisiting examples from Task A, there is no regularization signal to anchor the weights for the old task. The loss landscape for Task A is simply forgotten. Techniques that mitigate forgetting, like replay buffers or generative replay, work by artificially reintroducing old data (or synthetic proxies) to provide this anchoring signal during new learning.

This is the core theoretical framework describing the cause. A neural network must balance two competing needs:

• Plasticity: The ability to flexibly learn new information (adapt to Task B).
• Stability: The ability to retain previously learned information (remember Task A).

Standard stochastic gradient descent (SGD) is highly plastic but lacks stability. It rapidly converges to minima for the current task with no built-in mechanism to protect past knowledge. Catastrophic Forgetting is the extreme manifestation of unchecked plasticity. Mitigation strategies often explicitly manage this trade-off, for example, by elastic weight consolidation which slows down learning on weights deemed important for previous tasks.

A specific architectural point of failure is the final output layer. In a naive sequential learning setup, when switching from Task A to Task B, the output neurons are often repurposed. For example, a dog-vs-cat classifier (Task A) might have its output neurons overwritten to become car-vs-truck classifiers (Task B). This creates direct logit-level interference. Even if some internal feature representations remain intact, the mapping from those features to the final prediction is completely destroyed. Solutions include using progressive networks (adding new output heads) or dynamic architectures that expand to accommodate new tasks.

Forgetting occurs not just in the output layer but throughout the latent feature representations. Early convolutional layers learning basic edge and texture detectors may be somewhat stable, but higher-level, task-specific features are highly vulnerable. When learning Task B, the feature activations for Task A inputs can become distorted or disappear entirely. This is measured by the drift in activation patterns for old task samples. The phenomenon is related to representational drift in neuroscience. It demonstrates that catastrophic forgetting is a holistic network failure, not merely an output mapping problem.

An architectural method that freezes the parameters of a network trained on a previous task and instantiates a new, separate network (a "column") for each new task. Lateral connections from old columns to the new column allow the transfer of learned features without interference.

• Mechanism: Expands model capacity horizontally with each new task.
• Key Concept: Complete isolation of task-specific parameters prevents overwriting.
• Trade-off: Leads to linear growth in parameters and compute with the number of tasks.

A biologically-inspired technique where a rehearsal buffer stores a subset of data from previous tasks. During training on a new task, this stored data is interleaved with new examples.

• Mechanism: Mimics hippocampal replay by periodically retraining on old data.
• Implementation: Can use raw data, compressed features, or generated synthetic samples.
• On-Device Consideration: Buffer management is critical due to limited device memory; often employs reservoir sampling or generative replay.

Techniques that allocate distinct sub-networks or binary masks within a shared model for different tasks. PackNet and HAT (Hard Attention to the Task) are prominent examples.

• Mechanism: Learns a binary mask per task that gates which parameters are active.
• Key Concept: A single model's capacity is partitioned dynamically, preventing parameter overlap and interference.
• Efficiency: Highly parameter-efficient compared to progressive networks, as the base model is reused.

Continual Learning (or Lifelong Learning) is the machine learning paradigm where a model learns sequentially from a non-stationary stream of data or tasks over its lifetime. The primary objective is to accumulate knowledge without catastrophic forgetting, enabling the system to adapt to new information while preserving performance on previously encountered data. This is distinct from traditional batch learning, which assumes all training data is available simultaneously. Core challenges include managing task boundaries, data distribution shifts, and balancing stability (retaining old knowledge) with plasticity (acquiring new knowledge).

Elastic Weight Consolidation is a regularization-based algorithm designed to mitigate catastrophic forgetting in neural networks. It operates on the principle that not all network parameters (weights) are equally important for previously learned tasks. EWC estimates the Fisher information matrix to identify which weights are most critical for old tasks and then penalizes changes to those weights during new training. This creates an elastic constraint, allowing less important weights to change more freely. EWC is a foundational method for task-incremental learning, where the task identity is known at inference time.

Gradient Episodic Memory is an algorithmic approach that stores a subset of representative examples from past tasks in a constrained episodic memory buffer. When learning a new task, GEM computes the gradient for the new data and projects it so that the loss on the examples in the episodic memory does not increase. This is formulated as a constrained optimization problem, ensuring that updates to the model do not negatively impact performance on past tasks. GEM and its variant, Averaged GEM (A-GEM), provide a more direct guarantee against forgetting compared to pure regularization methods.

Experience Replay is a biologically-inspired technique where a model interleaves training on new data with rehearsals on stored examples from previous tasks. A replay buffer retains a small, often fixed-size, set of past data samples or their feature representations. During training on a new task, batches are constructed by mixing new data with samples drawn from this buffer. This approach helps maintain the decision boundaries for old classes and mitigates representational drift. Strategies include:

• Random Selection: Uniformly sampling from stored data.
• Reservoir Sampling: Maintaining a statistically representative buffer over a stream.
• Generative Replay: Using a generative model to produce synthetic samples of past data.

Parameter Isolation methods, also known as architectural strategies, combat catastrophic forgetting by dynamically allocating dedicated model capacity (sub-networks or parameters) for each new task. Instead of sharing all weights, these methods expand or mask the network to preserve the frozen parameters used for previous tasks. Key implementations include:

• Progressive Neural Networks: Introduce new columns of parameters for each task, with lateral connections to previous columns.
• PackNet: Iteratively prunes and freezes a subset of weights important for a task, then uses the remaining capacity for the next task.
• HAT (Hard Attention to the Task): Uses a soft attention mask over network activations that can be hardened to protect task-specific pathways. These methods often achieve very low forgetting but at the cost of increased model size or complexity.

Class-Incremental Learning is a stringent benchmark scenario within continual learning where a model must learn to classify new classes over time without being explicitly told the task identity during inference. The model's output space expands with each new set of classes, and it must distinguish between all classes seen so far. This setting is particularly challenging because it combines the problem of catastrophic forgetting with the need for a unified classifier. It tests an algorithm's ability to maintain a stable representation for old classes while integrating new ones, and often requires techniques like a dynamically expanding output layer, knowledge distillation from a previous model, and robust rehearsal strategies.