Catastrophic Forgetting

Catastrophic forgetting is the direct manifestation of the stability-plasticity dilemma, a fundamental trade-off in neural networks. Plasticity is the network's ability to learn new patterns by updating its weights. Stability is its ability to retain previously encoded knowledge. Standard gradient-based training optimizes for plasticity on the current data batch, inherently overwriting weights crucial for old tasks, thus destabilizing prior knowledge. This dilemma is the root cause that continual learning algorithms must explicitly address.

The core technical driver is parameter interference or catastrophic interference. In a densely connected neural network, most weights are shared (overlapping) across tasks. When gradients for a new task are applied, they indiscriminately change these shared parameters. Changes that are beneficial for the new task are often detrimental to the functional mappings required for old tasks. This interference is non-linear and compounded across layers, leading to rapid, non-graceful performance collapse rather than a slow decline.

Forgetting manifests differently based on the learning scenario:

• Task-Agnostic (Class/Domain-Incremental): The most challenging form. The model must learn new classes or domains over time without being told which task a sample belongs to at test time. Forgetting here is a complete corruption of the unified representation space.
• Task-Aware (Task-Incremental): The model receives an explicit task identifier. Forgetting here often relates to losing the ability to perform the task-specific computation, even if the identifier is provided, due to overwritten feature extractors or decision boundaries. The lack of a task ID amplifies the difficulty of mitigating interference.

Catastrophic forgetting is often asymmetric and irreversible under standard training. Learning Task B can severely degrade performance on Task A, but returning to train on Task A can then degrade performance on Task B, creating a destructive ping-pong effect. This is because gradient steps are not commutative. The loss landscape for the old task is permanently altered by the new training trajectory. Simply rehearsing on old data is required to reverse damage, but full recovery is not guaranteed and is computationally costly.

The severity of forgetting is heavily influenced by the semantic similarity or relatedness between sequential tasks.

• High Similarity (Positive Forward Transfer): Learning a related new task (e.g., different dog breeds) may initially improve performance on the old task due to shared features, though interference can still occur later.
• Low Similarity/High Conflict: Learning a dissimilar or contradictory task (e.g., 'dog' vs. 'cat' classification with overlapping features) causes severe negative backward transfer, as the network is forced to repurpose the same neurons for conflicting functions, maximizing interference.

Ironically, properties that make deep learning powerful also exacerbate catastrophic forgetting:

• High Model Capacity & Overparameterization: Enables fitting new tasks perfectly, providing no inherent pressure to preserve old functions.
• Stochastic Gradient Descent (SGD): The local, batch-wise optimization has no memory of past data distributions.
• Distributed Representations: Knowledge is encoded across many overlapping weights, making it fragile to interference.
• Batched Training: Standard practice assumes independent and identically distributed (IID) data, a condition violently violated in continual learning streams.

Continual learning is the overarching machine learning paradigm where a model learns sequentially from a non-stationary stream of data or tasks. Its primary objective is to accumulate knowledge over time without suffering from catastrophic forgetting. It encompasses several key scenarios:

• Task-Incremental Learning: Tasks are presented sequentially with an explicit task identifier provided at test time.
• Class-Incremental Learning: New classes are introduced over time, and the model must discriminate between all seen classes without a task ID.
• Domain-Incremental Learning: The input data distribution shifts, but the underlying output classes remain the same.
• Online Continual Learning: The strictest setting where the model receives a single, non-repeating pass over a data stream.

The stability-plasticity dilemma is the fundamental, competing tension at the heart of continual learning and catastrophic forgetting. It describes the challenge of balancing two opposing needs:

• Stability: The model's ability to retain consolidated knowledge and maintain performance on previously learned tasks.
• Plasticity: The model's capacity to adapt flexibly and learn new information from incoming data. All continual learning algorithms are, in essence, different engineering solutions to this core trade-off. Too much stability leads to intransigence (inability to learn new things), while too much plasticity leads to catastrophic forgetting.

Elastic Weight Consolidation (EWC) is a seminal regularization-based algorithm for mitigating catastrophic forgetting. It operates by identifying which parameters (synaptic weights) in a neural network are most important for previous tasks and making them "stiff" to change.

• Mechanism: After learning a task, EWC calculates an importance weight for each parameter, typically using the diagonal of the Fisher information matrix. When learning a new task, a quadratic penalty term is added to the loss function, penalizing changes to important parameters.
• Analogy: It functions like an elastic tether on each weight, with the importance determining the tether's strength, allowing less important weights to move more freely.

Experience Replay (ER) is a rehearsal-based strategy that directly combats catastrophic forgetting by storing a subset of past training examples in a fixed-size replay buffer. During training on new data, the model interleaves batches from this buffer.

• Core Function: By periodically "replaying" old data, the model rehearses previous tasks, preventing its parameters from drifting too far from solutions that worked for past data.
• Variants: Includes Dark Experience Replay (DER), which stores model logits for stronger consistency loss, and Generative Replay, where a generative model produces synthetic past data.
• Key Challenge: Managing buffer sampling strategies and size to balance retention efficiency with memory constraints.

Gradient Episodic Memory (GEM) is a constraint-based continual learning algorithm. Instead of replaying data, it stores past examples in an episodic memory and uses them to define constraints on gradient updates for new tasks.

• Mechanism: Before applying a gradient step for a new task, GEM computes the gradient on the episodic memory. It then projects the proposed new gradient onto a direction that does not increase the loss on the remembered past examples. This is a gradient projection operation.
• Goal: Ensure that learning the new task causes positive or neutral backward transfer, never negative transfer (forgetting).
• Advantage: Provides a formal guarantee against an increase in past task loss, given the constraints of the finite memory.

These are the two primary quantitative metrics for evaluating continual learning algorithms beyond simple accuracy, measuring knowledge flow across tasks.

• Forward Transfer: Measures how learning a previous task improves the initial performance or learning speed on a new, related task. It indicates positive knowledge generalization from past to future.
• Backward Transfer: Measures how learning a new task affects performance on previously learned tasks.
  • Positive Backward Transfer: The new task improves understanding of an old task (e.g., learning French improves Spanish).
  • Negative Backward Transfer (Interference): The new task degrades performance on an old task—this is the quantitative measure of catastrophic forgetting. A successful algorithm maximizes forward and positive backward transfer while minimizing negative backward transfer.