Catastrophic Forgetting

The most defining characteristic is the sudden and severe drop in accuracy on previously mastered tasks. This is not a gradual decline but a sharp, often near-total, loss of capability. The model's internal representations for old tasks are overwritten by the new learning signal.

• Example: A model trained to classify cats vs. dogs, then fine-tuned on cars vs. trucks, may see its cat/dog accuracy plummet from 95% to near-random (50%) after the second training phase.

Forgetting occurs because neural networks use shared, overlapping parameters (weights) for all tasks. During gradient descent on new data, updates optimized for the new task are misaligned with the optimal configuration for old tasks. This causes unlearning as weights drift from their previous values.

• The core issue is multitask interference within a fixed-capacity network. Without constraints, learning a new task B directly degrades the model for task A.

Catastrophic forgetting is the extreme manifestation of the stability-plasticity trade-off. A model must be plastic to learn new information but stable to retain old knowledge. Standard neural networks are highly plastic but lack inherent stability mechanisms.

• Stability refers to resistance to change (retaining old knowledge).
• Plasticity refers to adaptability (acquiring new knowledge).
• Catastrophic forgetting occurs when plasticity dominates, completely destabilizing prior learning.

Forgetting is triggered by a significant shift in the training data distribution between sequential learning phases. The severity correlates with the dissimilarity between old and new tasks/data.

• Sequential Task Learning: Training on Task A, then Task B (e.g., digit classification, then letter classification).
• Non-Stationary Data Streams: The underlying data distribution changes over time (e.g., user preference drift, seasonal trends).
• The model interprets the new data as the only relevant signal, causing it to specialize at the expense of generality.

In standard supervised training, a model is exposed to the entire dataset for all tasks simultaneously, allowing it to find a joint optimum. Catastrophic forgetting occurs in sequential learning settings where past data is unavailable for rehearsal.

• The model cannot revisit or reinforce patterns from previous tasks.
• This violates the independent and identically distributed (I.I.D.) data assumption central to most training paradigms, leading to non-stationary optimization.

The phenomenon is most acute in deep feedforward and recurrent networks with dense, shared parameters. Some architectures are more prone than others:

• Highly Parameterized Networks: More shared weights increase interference potential.
• Softmax Output Layers: A fixed-size output layer forces competition for capacity between old and new classes in incremental learning.
• Contrast with modular or sparse networks, which can isolate task-specific components, offering inherent resistance.

Continual learning (lifelong or incremental learning) is the overarching machine learning paradigm where a model learns sequentially from a non-stationary stream of data or tasks. The core objective is to accumulate knowledge over time while maintaining performance on previously encountered data, directly confronting the challenge of catastrophic forgetting.

• Key Scenarios: Includes task-incremental, domain-incremental, and class-incremental learning.
• Core Challenge: Managing the plasticity-stability trade-off—balancing the ability to learn new concepts (plasticity) with the ability to retain old ones (stability).

The plasticity-stability trade-off is the fundamental, competing tension at the heart of continual learning and neuroscience. Plasticity refers to a neural network's capacity to adapt its parameters to learn new information. Stability refers to its resistance to changing parameters encoding old knowledge.

• Catastrophic Forgetting is the extreme failure of stability.
• Over-regularization for stability can lead to catastrophic interference, where a model becomes unable to learn new tasks.
• Effective algorithms explicitly manage this trade-off, such as Elastic Weight Consolidation (EWC) which selectively penalizes changes to important parameters.

Experience replay is a biologically-inspired mitigation technique that stores a subset of past training examples in a memory buffer (or replay buffer) and interleaves them with new data during training. This rehearsal of old data prevents the model's parameters from drifting too far from solutions that work for previous tasks.

• Core Mechanism: Provides direct exposure to old data distributions.
• Memory Buffer Types: Can be fixed-size (FIFO, reservoir sampling) or dynamic.
• Related Technique: Generative replay or pseudo-rehearsal, where a generative model produces synthetic past data for replay, eliminating the need to store raw examples.

Elastic Weight Consolidation is a seminal regularization-based algorithm for mitigating catastrophic forgetting. It estimates the importance (Fisher information) of each model parameter to previously learned tasks. During training on a new task, EWC adds a quadratic penalty term to the loss function that heavily penalizes changes to parameters deemed important for old tasks, while allowing less important parameters to change freely.

• Analogy: Treats important parameters as being connected by "elastic bands" that resist movement.
• Key Innovation: Enables sequential learning without storing raw past data, only requiring a diagonal Fisher information matrix per task.
• Limitation: Assumes tasks are learned sequentially with clear boundaries and can suffer from parameter importance accumulation errors.

Learning without Forgetting is a foundational algorithm that uses knowledge distillation as a regularization mechanism for continual learning. When learning a new task, LwF uses the model's own predictions (logits) on the new task data—recorded before starting the new training—as soft targets to compute a distillation loss. This loss encourages the model to retain its original output behavior for the new data, thereby preserving knowledge related to old tasks.

• Key Advantage: Does not require storing any past training data (exemplar-free).
• Mechanism: Applies a distillation loss between the model's current and previous outputs on new data.
• Use Case: Particularly effective in task-incremental learning scenarios where task identity is known at inference.

Class-incremental learning is one of the most challenging and practical continual learning scenarios. The model must learn new classes incrementally over time from a stream of data, while maintaining a unified classifier that performs well on all classes seen so far. The model is not provided with task identifiers at test time, forcing it to distinguish between all classes.

• Core Difficulty: Must expand its output space and avoid catastrophic forgetting of past classes while learning new ones.
• Exemplar-based solutions: Algorithms like iCaRL combine a small episodic memory of stored examples with a nearest-mean-of-exemplars classification rule and knowledge distillation.
• Exemplar-free solutions: Rely heavily on regularization (e.g., LwF) or dynamic architectures to isolate knowledge.