Catastrophic Forgetting

Continual learning, also known as lifelong learning, is the machine learning paradigm where a model learns sequentially from a stream of data or tasks, aiming to accumulate knowledge over time without catastrophically forgetting previous information. It is the overarching field that directly addresses the problem of catastrophic forgetting.

• Core Objective: To develop systems that can adapt to new information while preserving past knowledge.
• Key Challenge: Balancing stability (retaining old knowledge) with plasticity (acquiring new knowledge).
• Approaches: Include regularization-based methods (e.g., Elastic Weight Consolidation), architectural methods (e.g., progressive neural networks), and replay-based methods (e.g., using a memory buffer of past examples).

Elastic Weight Consolidation (EWC) is a regularization-based algorithm designed to mitigate catastrophic forgetting by slowing down learning on the model parameters deemed most important for previously learned tasks.

• Mechanism: After learning a task, EWC calculates a Fisher information matrix to estimate the importance (or "elasticity") of each network weight. During training on a new task, a penalty term is added to the loss function that discourages significant changes to these important weights.
• Analogy: It functions like a spring, anchoring important parameters to their previous values.
• Use Case: Particularly effective for sequential task learning where task boundaries are clear. It is a foundational technique in the field of continual learning.

Experience replay is a technique, originally from reinforcement learning, adapted for continual learning to combat catastrophic forgetting by storing and intermittently retraining on a subset of past data.

• Core Process: A replay buffer stores a sampling of past experiences (e.g., input-label pairs or state-action-reward tuples). During training on new data, the model is also trained on mini-batches drawn from this buffer.
• Primary Benefit: It helps maintain the decision boundaries for old classes or tasks by providing interleaved exposure.
• Variants: Includes generative replay, where a generative model (like a GAN) is trained to produce synthetic samples of past data, eliminating the need to store raw data.
• Connection to Memory: Directly implements a form of episodic memory within the learning system.

Multi-task learning (MTL) is a learning paradigm where a single model is trained concurrently on multiple related tasks, sharing representations between them. It is often contrasted with sequential continual learning, as it avoids catastrophic forgetting by design.

• Key Principle: Leverages commonalities and differences across tasks to improve generalization and data efficiency.
• Architecture: Often uses hard parameter sharing (shared hidden layers with task-specific output heads) or soft parameter sharing (separate models with regularization to encourage similarity).
• Limitation: Requires simultaneous access to data from all tasks during training, which is often impractical in dynamic, real-world scenarios where data arrives sequentially.

The stability-plasticity dilemma is a fundamental trade-off in adaptive systems, including neural networks, between the ability to retain existing knowledge (stability) and the ability to integrate new information (plasticity). Catastrophic forgetting is the extreme failure case where plasticity completely overwhelms stability.

• Biological Origin: Coined in neuroscience to describe how the brain manages learning and memory.
• In AI Systems: A model that is too stable becomes incapable of learning anything new (rigid). A model that is too plastic will overwrite old knowledge with new information (catastrophic forgetting).
• Engineering Goal: The design of continual learning algorithms is essentially the search for an optimal balance point on this spectrum.

Forward and backward transfer are metrics used to evaluate continual learning systems, measuring how learning a new task affects performance on future and past tasks, respectively.

• Forward Transfer (FWT): The influence that learning a task has on the performance of a future, not-yet-learned task. Positive FWT indicates that earlier learning helped later learning.
• Backward Transfer (BWT): The influence that learning a new task has on the performance of a previously learned task. Catastrophic forgetting is evidenced by large negative backward transfer. Positive BWT indicates that learning a new task actually improved performance on an old one.
• Significance: These metrics provide a more nuanced view of model performance than average accuracy alone, directly quantifying knowledge retention and interference.