Elastic Weight Consolidation (EWC)

EWC quantifies the importance of each network parameter (weight) for a learned task using an approximation of the Fisher information matrix. The diagonal of this matrix estimates how much a change in a specific weight would affect the model's output log-likelihood. A high Fisher value indicates a weight is crucial for the task; EWC will strongly penalize changing it when learning a new task. This provides a principled, data-driven alternative to heuristic importance measures.

The core mechanism of EWC is a regularization term added to the loss function when learning a new task (Task B). This term is a quadratic penalty centered on the optimal parameters from the previous task (Task A). The penalty for each weight is scaled by its computed importance (Fisher diagonal). The loss function becomes:

L(θ) = L_B(θ) + Σ_i (λ/2) * F_i * (θ_i - θ*_A,i)^2

Where λ is a hyperparameter controlling the strength of consolidation, F_i is the Fisher importance for weight i, and θ*_A,i is the optimal value for weight i after learning Task A. This elastic constraint allows less important weights to change more freely.

EWC operates in a task-incremental or domain-incremental learning setting, where tasks are presented sequentially. Crucially, the algorithm does not require an explicit task identifier at inference time. The consolidated network is a single model that can perform all learned tasks. This is different from multi-head architectures that require task-specific output layers. The trade-off is that EWC's capacity is fixed; performance can degrade as the number of tasks grows, as the parameter space becomes increasingly constrained.

Calculating the full Fisher information matrix is computationally prohibitive for large models. EWC uses a key approximation: it only uses the diagonal of the Fisher matrix, assuming parameter importances are independent. This makes the method scalable but ignores correlations between weights. The Fisher is typically estimated using a single pass over the training data for the old task, often just the training set labels under the model's predictive distribution. This makes EWC more efficient than rehearsal-based methods like Generative Replay, which require storing data or training a generator.

EWC has a theoretical foundation in Bayesian inference. Learning Task A provides a prior distribution over the model parameters, P(θ|D_A). Learning Task B then involves computing the posterior P(θ|D_A, D_B). EWC approximates this posterior by assuming the prior P(θ|D_A) is a Gaussian distribution centered at θ*_A with a diagonal precision matrix given by the Fisher information. The quadratic penalty term is derived from the log of this Gaussian prior. This perspective frames continual learning as sequential Bayesian updating.

While foundational, EWC has known limitations:

• Diagonal Approximation: Ignoring parameter correlations can lead to suboptimal importance estimates.
• Fixed Network Capacity: As more tasks are added, the shared parameter space becomes a bottleneck, leading to intransigence (inability to learn new tasks).
• Memory Overhead: Requires storing the Fisher diagonal and optimal parameters (θ*) for each previous task, leading to linear growth in memory with tasks.
• Sensitivity to Hyperparameter λ: The consolidation strength must be tuned. These limitations inspired subsequent algorithms like Synaptic Intelligence (SI) and Memory Aware Synapses (MAS).

In reinforcement learning, an agent must master a sequence of tasks (e.g., different video game levels or robotic manipulation skills). Applying EWC involves:

• Training a policy network on the first environment.
• Computing the parameter importance based on the agent's performance.
• Using this importance to constrain learning when the agent is deployed in a new, distinct environment. This prevents the agent from forgetting how to solve earlier tasks, enabling the development of more generalist, multi-skilled agents without the need for separate networks per task.

EWC is applied to language models that need to adapt to new domains or tasks over time without retraining from scratch. For example, a model fine-tuned for legal document analysis could later be adapted for medical report summarization. EWC identifies which weights in the transformer layers are crucial for the legal domain and restricts their movement during medical training. This preserves the core linguistic understanding and legal expertise while integrating new medical terminology and reasoning patterns, creating a more versatile model.

EWC is a key enabler for federated continual learning on edge devices like smartphones. A pre-trained model (e.g., for next-word prediction) is deployed. As the user interacts with the device, the model learns personalized patterns (e.g., unique slang, frequent contacts). EWC ensures this personalization does not corrupt the model's general language knowledge. Only a diagonal approximation of the Fisher matrix is stored per user, which is computationally cheap and privacy-preserving, as raw personal data never leaves the device.

A robotic arm learning a sequence of motor skills (e.g., grasping, pushing, placing) can use EWC to consolidate knowledge. After learning to grasp various objects, the importance weights for the neural controller are frozen. When learning to push objects, the network is penalized for altering these crucial grasping parameters. This allows the robot to build a composite skill set where later learning does not degrade earlier, more fundamental abilities. It is particularly valuable for lifelong learning robots in unstructured environments.

While powerful, EWC has key constraints that dictate its use:

• Quadratic Memory Growth: The naive Fisher matrix scales with the square of the number of parameters, which is infeasible for large models. Practical implementations use a diagonal approximation, storing only one importance value per parameter.
• Task Identity Requirement: EWC requires explicit knowledge of when a task boundary occurs to compute the Fisher information. It is less suited for true online learning with no clear task delineation.
• Accuracy-Complexity Trade-off: The strength of the quadratic penalty (the lambda hyperparameter) must be tuned. Too weak leads to forgetting; too strong prevents learning new tasks (intransigence).

Generative Replay, also known as pseudo-rehearsal, is a continual learning paradigm where a generative model (e.g., a Generative Adversarial Network or Variational Autoencoder) is trained to produce synthetic data samples that approximate the distribution of previous tasks. During training on a new task, these generated samples are interleaved with real data from the new task, simulating experience replay without the need to store raw data. This contrasts with EWC's parameter regularization approach.