Catastrophic Forgetting

The core mechanism is the unconstrained overwriting of shared model weights. During gradient descent, weight updates for a new task move parameters to a configuration optimal for that task, erasing the configuration that was optimal for prior tasks. This is not a failure of memory but of interference in a shared parameter space.

• Example: A convolutional network trained to classify cats, then dogs, will adjust the same early-layer filters that previously detected generic animal edges, causing them to specialize for dog-specific features and lose cat-specific sensitivity.

Standard neural networks are stateless with respect to task identity. During inference, the model receives an input but has no internal signal indicating which task it should perform. The same forward pass is used for all learned tasks, forcing the network to find a single set of weights that works for everything—an often impossible compromise leading to forgetting.

This contrasts with modular architectures or systems with an external task selector, which can route inputs to specialized sub-networks.

SGD and its variants are inherently forgetful. They optimize for immediate loss reduction on the current mini-batch, with no mechanism to preserve performance on data not in the current training distribution. The plasticity required to learn new patterns directly enables the erasure of old ones.

• Key Insight: The learning process has no concept of stability (retaining old knowledge) versus plasticity (acquiring new knowledge). This is known as the stability-plasticity dilemma.

Forgetting is most severe when tasks share representational overlap—using similar features or neural pathways. High overlap causes maximal negative backward transfer, where learning the new task actively degrades performance on the old one.

• Low-Overlap Example: Learning to classify images, then audio signals, may cause less forgetting if the model has separate processing streams.
• High-Overlap Example: Learning to classify two similar bird species will cause severe interference in shared feature extractors.

The defining condition is the non-stationary data stream. In continual learning, the model never sees old task data again during training on new tasks. This violates the core independent and identically distributed (IID) assumption of standard supervised learning, under which SGD is designed to converge to a single, stationary optimum.

The model's loss landscape shifts with each new task, and SGD simply follows the gradient downhill on the new landscape, falling out of the valley that was optimal for the old task.

The phenomenon is not unique to deep networks. It was formally identified in 1989 by McCloskey and Cohen in simple two-layer linear networks. Their work showed that training a network on the A-B association, then on A-C, completely abolishes the ability to recall A-B. This proves the cause is fundamental to distributed representation in connectionist models, not depth or non-linearity.

• Implication: The problem is architectural and algorithmic, not a byproduct of modern deep learning complexity.

Continual Learning (or Lifelong Learning) is the paradigm where a model learns sequentially from a non-stationary stream of data, acquiring new knowledge over time. The primary objective is to maintain performance on previously learned tasks while integrating new information, directly opposing catastrophic forgetting.

• Key Challenge: Balancing stability (retaining old knowledge) with plasticity (acquiring new knowledge).
• Common Approaches: Include rehearsal (replaying old data), regularization (penalizing changes to important weights), and architectural methods (adding new network components).
• On-Device Relevance: Essential for IoT sensors and personal devices that must adapt to user behavior or environmental changes without cloud retraining.

Elastic Weight Consolidation is a regularization-based algorithm designed to mitigate catastrophic forgetting. It identifies which parameters (weights) in a neural network are most important for previously learned tasks and penalizes changes to them during new training.

• Mechanism: Calculates a Fisher Information Matrix to estimate parameter importance. A quadratic penalty term is added to the loss function, making important weights "elastic"—they can change, but at a high cost.
• Advantage: Enables sequential learning without storing or replaying old raw data, preserving privacy.
• Limitation: Assumes tasks are learned sequentially and requires calculating importance metrics, which can be computationally intensive for large models.

Gradient Episodic Memory is an optimization-based approach that constrains gradient updates to prevent interference with past knowledge. It stores a subset of past training examples (an episodic memory) and uses them to define constraints on the loss for new tasks.

• Core Process: When computing gradients for a new task, GEM projects them onto a direction that does not increase the loss on examples from the episodic memory. If the projection fails, it performs a corrective update.
• Benefit: Provides a strong theoretical guarantee against negative backward transfer (forgetting).
• Trade-off: Requires maintaining a fixed-size memory buffer of old data, which may not be feasible for all on-device scenarios due to storage limits.

Progressive Neural Networks are an architectural solution to catastrophic forgetting. Instead of updating a single network, a new, separate column (sub-network) is instantiated for each new task. These columns are connected via lateral connections that allow the new column to leverage features from previous columns.

• Key Principle: No forgetting by design, as old network parameters are frozen.
• Advantages: Perfect retention of prior task performance and positive forward transfer (new tasks can benefit from old features).
• Disadvantages: Network capacity grows linearly with the number of tasks, making it computationally and memory inefficient for long task sequences—a critical concern for TinyML deployment.

Experience Replay is a biologically-inspired technique where a model is periodically retrained on a mixture of new data and a small, stored subset of data from previous tasks. This interleaving of old and new examples during training helps maintain decision boundaries for past classes.

• Implementation: Uses a fixed-size memory buffer (e.g., a ring buffer) to store representative samples. During training on a new task, batches are constructed by sampling from both the current data and this buffer.
• On-Device Consideration: The memory buffer size is a critical trade-off between performance and the limited RAM of microcontrollers. Advanced strategies include coreset selection to store the most informative examples.
• Connection: A foundational component in many rehearsal-based continual learning algorithms.

This approach uses meta-learning (learning to learn) to train models with an inherent bias against catastrophic forgetting. The model is meta-trained on a distribution of sequential learning problems so it can quickly adapt to new tasks with minimal interference.

• Objective: Learn an initialization or an optimization algorithm that is inherently robust to sequential training.
• Example: Model-Agnostic Meta-Learning (MAML) can be adapted to find weight initializations from which a model can be fine-tuned to a new task in just a few steps, causing minimal disturbance to performance on prior tasks.
• Promise: Offers a path to more general and efficient continual learning agents, though meta-training itself is computationally expensive and typically performed offline before on-device deployment.