Catastrophic Forgetting

A synaptic stabilization algorithm that identifies and protects parameters critical to previously learned tasks. EWC calculates the Fisher Information Matrix to quantify the importance of each weight. During subsequent training, a quadratic penalty term is added to the loss function, constraining changes to high-importance weights. This allows the network to learn new tasks while performance on old tasks remains stable, mimicking the neural consolidation observed in biological brains.

A rehearsal-based method that interleaves samples from a memory buffer of previous tasks with new data during training. By periodically retraining on stored historical examples, the model maintains its decision boundaries for old classes. Key variants include:

• Reservoir sampling: Maintains a fixed-size buffer with uniform random replacement.
• Generative replay: Uses a separate generative model to synthesize pseudo-samples of past data instead of storing raw examples, preserving privacy.

An architectural approach that instantiates a new neural network column for each new task while freezing previously learned columns. Lateral connections from frozen columns to the new column enable forward transfer of learned features without modifying old parameters. This completely eliminates forgetting at the cost of linear growth in model parameters with each task, making it suitable for environments with abundant compute and a finite, known sequence of tasks.

A functional regularization technique where the model being trained on new data is constrained to match the soft output probabilities of a frozen copy of its former self on old tasks. The Kullback-Leibler divergence between the old and new model's logits is added to the loss function. This preserves the relational structure of the original output space without requiring access to historical raw data, making it ideal for privacy-sensitive federated settings.

An online, task-agnostic regularization method that computes per-synapse importance measures during training without requiring explicit task boundaries. SI tracks the accumulated gradient path of each weight and assigns high importance to synapses that contributed significantly to loss reduction. A surrogate loss then penalizes deviation from these consolidated values, enabling continual learning in streaming data environments where task switches are not explicitly signaled.

A system-level strategy combining FedProx or SCAFFOLD with local replay buffers to manage non-IID data streams across a fleet. The global model is periodically distilled against a proxy dataset or a frozen checkpoint. Techniques include:

• Server-side rehearsal: A curated global buffer of representative samples.
• Gradient alignment: Projecting local updates to avoid interference with the global objective direction. This prevents catastrophic forgetting at both the local client and global server levels.