Catastrophic Forgetting Avoidance

Elastic Weight Consolidation (EWC) is a regularization-based method that mitigates catastrophic forgetting by identifying and protecting the most important parameters for previously learned tasks. It adds a penalty term to the loss function during new task training.

• Mechanism: Calculates a Fisher Information Matrix to estimate parameter importance. Parameters crucial for past tasks are assigned a high consolidation strength, making them resistant to large changes.
• Federated Application: The server can compute a global importance matrix by aggregating client contributions, then distribute it to guide local training on new tasks, ensuring shared knowledge is preserved.
• Example: A federated model for medical imaging that first learns to detect pneumonia must use EWC when later learning to detect fractures, preventing its pneumonia detection accuracy from collapsing.

Experience Replay (ER) is a rehearsal-based technique where a subset of data from previous tasks is stored and interleaved with new task data during training to remind the model of past knowledge.

• Core Principle: Maintains a fixed-size replay buffer of representative samples or their embeddings from past learning rounds.
• Federated Implementation: Can be implemented via a server-managed global buffer containing aggregated, privacy-preserving representations, or through client-managed local buffers. During a new federated round, clients train on a mix of new local data and sampled data from the buffer.
• Challenge: Balancing buffer size with privacy and communication costs. Techniques like coreset selection or generative replay (using a generative model to produce synthetic past data) are often used to improve efficiency.

Gradient Episodic Memory (GEM) is an optimization-based approach that directly constrains the gradient updates for a new task to not increase the loss on past tasks.

• Process: Stores a small episodic memory of examples from previous tasks. When computing gradients for a new mini-batch, GEM projects these gradients onto a feasible region defined by the gradients of the memory examples.
• Federated Adaptation: In a federated setting, each client can maintain a local episodic memory. The server's aggregation must account for these constrained updates. Alternatively, a global memory can guide client updates to ensure the consolidated model does not forget.
• Advantage: Provides a stronger guarantee than penalty-based methods like EWC, as it actively prevents negative backward transfer by ensuring loss on past tasks does not rise.

This family of techniques avoids interference between tasks by allocating distinct model parameters or expanding the network architecture for each new task.

• Key Methods:
  • PackNet: Identifies and freezes a subset of important weights for each learned task, then uses remaining capacity for new tasks.
  • Progressive Neural Networks: Adds a new column of parameters (a new network) for each task, with lateral connections to previous columns to facilitate transfer.
  • Adapter Modules: Inserts small, task-specific trainable modules (like LoRA layers) into a frozen base model.
• Federated Consideration: These methods are highly compatible with personalization. Clients can learn task-specific adapters or masks on a shared frozen backbone, eliminating cross-client interference. The server only aggregates adapter weights, not the core model, enhancing stability.

Meta-Learning for Continual Learning trains a model's learning algorithm or initialization to be inherently resilient to forgetting when faced with a stream of tasks.

• Objective: Learn an initialization or an optimizer that can quickly adapt to a new task with minimal data (few-shot) while retaining performance on old tasks. This is often framed as a meta-objective over a sequence of tasks.
• Federated Formulation (Meta-Fed): The server meta-trains a global model initialization across many clients, each representing a different task or data distribution. The goal is for this initialization to allow each client to personalize effectively without forgetting its local historical tasks after aggregation.
• Benefit: Moves the burden of catastrophic forgetting avoidance from the training-time algorithm to the learned model prior, creating a more fundamentally adaptable system.

This technique uses Knowledge Distillation to transfer the "knowledge" of a model trained on previous tasks (the teacher) to the model being trained on a new task (the student), thereby preserving old behaviors.

• Mechanism: Instead of just using ground-truth labels for the new task, the loss function includes a distillation loss term that penalizes deviations from the teacher model's output probabilities (soft targets) or intermediate representations.
• Federated Application: The server maintains a copy of the global model from the previous federated round as the teacher model. During a new round, clients train their local models (students) on new data with a combined loss: task loss on new data + distillation loss aligning with the old global model's predictions.
• Outcome: The aggregated model retains functional behavior from past rounds while integrating new knowledge, smoothing the transition and reducing forgetting.

Elastic Weight Consolidation is a regularization-based method that prevents catastrophic forgetting by identifying and protecting the most important parameters for previously learned tasks. It calculates a Fisher Information Matrix to estimate parameter importance, then adds a quadratic penalty term to the loss function during new task training. This penalty discourages large changes to critical weights, effectively anchoring the model's knowledge.

• Key Mechanism: Adds a regularization term: L_total = L_new + λ/2 * Σ_i F_i (θ_i - θ_i)^2, where F_i is the importance for parameter i, and θ_i is its value after the previous task.
• Federated Application: In Continual Federated Learning, the server can maintain a global importance matrix aggregated from clients to guide consolidation during global model updates.

Experience Replay mitigates forgetting by storing a subset of representative data samples (or their embeddings) from previous tasks in a replay buffer. During training on a new task, these stored examples are interleaved with new data, periodically reminding the model of past patterns.

• Implementation Variants: Includes i.i.d. sampling from the buffer or more sophisticated herding to select prototypical samples.
• Federated Challenge: Direct data storage conflicts with federated learning's privacy principles. Solutions include:
  • Storing synthetic data generated by a generative model.
  • Using federated embeddings where only model activations are replayed.
  • Implementing pseudo-rehearsal by replaying the model's own generated outputs for past task inputs.

Gradient Episodic Memory is a constrained optimization approach. It stores a small episodic memory of past task data and, when computing gradients for a new task, projects them so they do not increase the loss on the memory examples. This ensures the model's updates are forward-transfer compatible.

• Core Constraint: Solves: minimize L(f_θ, new_task) subject to L(f_θ, memory_k) ≤ L(f_θ_old, memory_k) for all past tasks k.
• Advantage: Provides a hard guarantee against negative backward transfer (forgetting) on the memorized examples.
• Federated Consideration: The episodic memory must be constructed in a privacy-preserving manner, potentially using gradients or feature representations instead of raw client data.

This family of techniques allocates dedicated model parameters for each new task, physically isolating knowledge to prevent interference.

• Progressive Neural Networks: Introduce new columns of parameters for each task, with lateral connections to previous columns to enable feature transfer.
• PackNet: Iteratively prunes and freezes important weights for a task, then uses the freed capacity for the next task.
• Federated Relevance: In a federated setting, this can map to personalized layers where global base features are shared, but task-specific head networks are allocated per client or task group. The server must orchestrate the routing and aggregation of these specialized components.

Meta-learning frameworks like Model-Agnostic Meta-Learning (MAML) train a model's initialization to be highly adaptable. The model learns how to learn new tasks quickly from few examples, which inherently reduces the magnitude of weight updates needed and lessens forgetting.

• Process: The model is meta-trained on a distribution of tasks. For a new task, it performs a few gradient steps from the meta-initialized point.
• Federated Meta-Learning (FedMeta): The server learns a global meta-initialization by aggregating client updates, each of which is a meta-update computed over that client's local task distribution. This prepares the model for rapid, non-destructive adaptation to any client's new data.

This method uses knowledge distillation to transfer soft knowledge from a past model (the teacher) to the current model (the student) being trained on a new task. The student is trained to match both the new task labels and the softened output distributions (logits) of the teacher model for new data, preserving its original reasoning.

• Loss Function: L = α * L_CE(y_new, y_pred) + β * L_KL(σ(z_teacher/T), σ(z_student/T)), where T is a temperature scalar and σ is the softmax function.
• Federated Application: In Continual Federated Learning, the server can maintain a snapshot of the global model from the previous training round as the teacher. Clients use this snapshot during local training on new data to distill prior global knowledge into their updates before sending them to the server.

Elastic Weight Consolidation (EWC) is a regularization-based method to mitigate catastrophic forgetting. It identifies parameters critical to previous tasks by estimating their importance via the Fisher information matrix. During training on a new task, EWC applies a quadratic penalty that discourages large changes to these important parameters, effectively anchoring the model to its prior knowledge.

• Core Mechanism: Adds a regularization term to the loss function: L_total = L_new + λ * Σ_i F_i * (θ_i - θ*_i)^2, where F_i is the Fisher importance for parameter i, and θ*_i is its optimal value from the previous task.
• Use Case: Particularly effective in federated learning where clients learn sequential tasks locally, as it provides a mathematically grounded way to protect consolidated global model knowledge.

Experience Replay is a biologically-inspired technique that stores a subset of data (or generated representations) from previous tasks in a fixed-size buffer. During training on a new task, the model is periodically shown samples from this replay buffer, interleaving old and new knowledge to prevent forgetting.

• Core Variants: Includes i) Raw Data Replay (stores actual samples, privacy-sensitive), ii) Generative Replay (uses a generative model to produce pseudo-samples of old data), and iii) Representation Replay (stores and replays intermediate feature vectors).
• Federated Consideration: In decentralized settings, raw data replay is often infeasible due to privacy. Generative or representation replay, where only model-generated data or features are shared, becomes a critical alternative.

Continual Federated Learning (CFL) is the overarching paradigm where a global model is trained sequentially on a stream of non-stationary tasks across distributed clients. The primary objective is to accumulate knowledge over time while avoiding catastrophic forgetting, without centralized data storage.

• Key Challenge: Combines the difficulties of non-IID data across clients with temporal task shifts. The model must adapt to new data distributions while preserving performance on tasks learned from clients that may no longer be active.
• System Design: Requires orchestration of client selection, adaptive aggregation (e.g., weighting clients based on task relevance), and integration of forgetting avoidance techniques like EWC at the server or client level.

Gradient Episodic Memory (GEM) is an optimization-based approach that constrains updates for a new task to not increase the loss on previous tasks. It stores a small episodic memory of examples from past tasks and uses their loss gradients to define a feasible region in parameter space for the new update.

• Core Algorithm: When computing a gradient for a new task, GEM projects it to the closest direction (in gradient space) that does not increase the loss on the episodic memory. This is solved as a quadratic programming problem.
• Advantage: Provides a stronger guarantee than regularization-based methods—it aims for no forgetting, rather than just minimizing it. Its computational overhead can be a consideration in resource-constrained federated environments.

Parameter Isolation Methods avoid catastrophic forgetting by allocating distinct model parameters (e.g., specific neurons, attention heads, or entire layers) to different tasks. The knowledge for each task is physically separated within the model architecture.

• Common Techniques: Include i) Progressive Neural Networks: Add new columns of parameters for each new task, with lateral connections to previous columns for feature transfer. ii) PackNet: Iteratively prune and freeze parameters important for a task, then train remaining capacity on the next task.
• Federated Implication: These methods can lead to linear growth in model size with tasks, which is problematic for communication-efficient federated learning. Strategies for dynamic architecture expansion and compression across clients are an active research area.

Meta-Learning for Continual Learning involves training a model's initialization or its learning algorithm (the optimizer) on a distribution of sequential learning problems. The goal is to meta-learn an inductive bias that enables rapid adaptation to new tasks with minimal forgetting of old ones.

• Mechanism: Often framed as Model-Agnostic Meta-Learning (MAML) for continual learning. The meta-objective is to find an initialization such that a few gradient steps on a new task yield good performance without degrading performance on previous tasks, accessed via a replay buffer or task description.
• Federated Application: In Federated Meta-Learning, the server learns a global meta-initialization from the stream of tasks experienced by different clients. This initialization is inherently more resilient to forgetting when deployed to a client facing a new, related task.