Elastic Weight Consolidation (EWC)

EWC computes the Fisher information matrix on a dataset sampled from a previously learned task. This matrix quantifies how sensitive the model's output is to small perturbations in each weight.

• High Fisher Value: Indicates a weight is crucial for the prior task's performance; changing it would drastically increase the loss.
• Low Fisher Value: Indicates a weight is redundant or irrelevant to the prior task, allowing it to be freely repurposed for new learning.
• The diagonal of this matrix serves as the per-parameter importance weight in the EWC loss function.

EWC augments the standard loss function for a new task with a quadratic penalty term. This term penalizes the distance between the current weights and the optimal weights found for the previous task.

• The penalty is weighted by the Fisher information, creating an elastic anchoring effect.
• Parameters with high importance are held tightly in place, while unimportant parameters can move freely to minimize the new task's loss.
• This effectively creates a Gaussian approximation of the posterior distribution over weights for the old task.

Catastrophic forgetting is the phenomenon where a neural network abruptly and completely loses the ability to perform a previously mastered task upon training on a new one. EWC directly mitigates this.

• Standard gradient descent overwrites weights indiscriminately, destroying old knowledge representations.
• EWC's selective plasticity allows the network to find a shared representation that performs well on both tasks, provided sufficient capacity exists.
• It transforms the sequential learning problem into a constrained optimization problem, finding a low-error region in the weight space that is common to all tasks seen so far.

The algorithm is directly inspired by the neuroscience of memory retention in mammalian brains. It computationally mimics the process of synaptic consolidation.

• Biological Parallel: The neocortex is believed to consolidate memories by reducing the plasticity of synapses that are critical for retaining learned skills.
• Cascade Model: EWC acts like a computational cascade, where the Fisher information matrix identifies which artificial synapses to protect.
• This contrasts with rehearsal-based methods that require storing or generating raw data from previous tasks, which can be memory-intensive or privacy-violating.

A practical limitation of the original EWC is the need to store a separate Fisher matrix for every completed task, causing linear memory growth. Online EWC solves this.

• It consolidates all previous tasks into a single, running Fisher matrix.
• When a new task is finished, its Fisher information is combined with the existing consolidated matrix, often via a moving average.
• This reduces the memory overhead from O(N) to O(1) with respect to the number of tasks, making it viable for lifelong learning on edge devices with strict storage constraints.

The elasticity coefficient (λ) is a critical hyperparameter that balances the trade-off between stability and plasticity.

• High λ: The model is rigid, strongly preserving old knowledge but potentially failing to learn new tasks effectively.
• Low λ: The model is plastic, learning new tasks quickly but at a high risk of overwriting old representations.
• Tuning λ is essential and often requires a validation set that includes samples from both old and new tasks to find the optimal Pareto frontier between backward and forward transfer.

The fundamental problem that EWC is designed to solve. When a neural network is trained sequentially on new tasks, its performance on previously learned tasks degrades rapidly as the weights are overwritten to accommodate new knowledge. This occurs because standard gradient descent has no mechanism to protect parameters critical to earlier tasks. EWC directly addresses this by identifying and slowing learning on those crucial weights.

A continual learning method that estimates parameter importance based on the sensitivity of the learned function output to weight perturbations, rather than the loss gradient. MAS computes importance by accumulating the magnitude of gradients of the model's output with respect to each weight during unsupervised sampling. This makes MAS particularly effective when task boundaries are not clearly defined or when labeled data is scarce.