Continual Federated Learning

Catastrophic forgetting is the tendency of a neural network to abruptly and drastically lose previously learned knowledge when trained on new data. In CFL, this is exacerbated by data heterogeneity across clients and the inability to revisit old client data.

• Core Problem: A model learning Task B on Client 2 may overwrite weights critical for Task A learned earlier on Client 1.
• Mitigation Techniques: Methods like Elastic Weight Consolidation (EWC), which penalizes changes to important weights, and experience replay, where a small buffer of representative data from past tasks is stored on the server for rehearsal.

Client drift occurs when local models on different clients diverge significantly due to non-IID (Independent and Identically Distributed) data and sequential task exposure, making global aggregation unstable.

• Non-IID Sequential Data: Each client may experience a unique, non-stationary stream of tasks (e.g., User A's typing patterns change seasonally, while User B's app usage evolves differently).
• Impact: Standard Federated Averaging (FedAvg) performs poorly, as the single global model struggles to represent all historical tasks across drifting clients. Techniques like adaptive server optimizers (e.g., FedAdam) and controlled client updates are required.

CFL operates under the dual constraints of continuous learning and federated communication limits. Transmitting full model updates for every new task or rehearsal step is often infeasible.

• Bandwidth Cost: Rehearsing past tasks via experience replay may require additional communication rounds.
• On-Device Compute: Continual learning algorithms (e.g., EWC) add computational overhead to already constrained edge devices.
• Solutions: Employ sparse updates, federated distillation (sharing only logits), and efficient replay buffer management to minimize communication and compute footprints.

A fundamental challenge is task identification—determining when data from a client represents a new task versus a variation of a current one—without centralized data access.

• Server Blindness: The server only sees model updates, not raw data, making task boundary detection difficult.
• Orchestration Complexity: The server must decide when to trigger a new global learning phase, which clients to involve, and how to balance learning new tasks with consolidating old ones.
• Approaches: Use change detection in model updates or client-reported metadata to infer task shifts and dynamically adjust training regimes.

Mitigating catastrophic forgetting often requires replaying past data, which directly conflicts with federated learning's data minimization principle. Storing even a small amount of raw client data on the server creates a privacy risk.

• Privacy Conflict: Experience replay can become a privacy leak if the replay buffer contains sensitive client examples.
• Technical Solutions:
  • Generative Replay: Use a generative model on the server to create synthetic data approximating past distributions.
  • Federated Averaging of Memories: Clients maintain private replay buffers locally and learn from them during local training, never sharing the raw replay data.
  • Using Differential Privacy on aggregated updates related to past tasks.

Evaluating a CFL system is complex because performance must be measured across all learned tasks over time, for all clients, without centralized test sets.

• Multi-Task Evaluation: The global model must be assessed on Task 1, Task 2, ..., Task N after learning Task N, requiring carefully curated test sets for each.
• Fairness Across Clients & Time: A model may become biased toward recent tasks or clients with more frequent/diverse data streams. Performance on early tasks for inactive clients may degrade unfairly.
• Metrics: Requires tracking backward transfer (effect on past tasks), forward transfer (help for future tasks), and global accuracy matrix across tasks and client cohorts.

A regularization-based method to mitigate catastrophic forgetting. After learning a task, EWC estimates the importance (Fisher information) of each model parameter to the task. When learning a new task, it penalizes changes to parameters deemed important for previous tasks.

• Key Mechanism: Adds a quadratic penalty term to the loss function.
• Federated Adaptation: Clients compute local importance estimates, which the server aggregates to form a global importance mask.
• Use Case: Ideal for scenarios where task boundaries are clear and the server can maintain a consolidated importance matrix.

A constrained optimization approach that directly prevents updates that would increase loss on past tasks. It stores a small episodic memory of exemplars from previous tasks and uses them to define constraints during training on new data.

• Key Mechanism: Solves an optimization problem where the new task's gradient is projected to not violate loss constraints on memory samples.
• Federated Adaptation: Episodic memories are kept locally on clients or in a privacy-preserving form on the server. Gradient projections are performed during local training.
• Benefit: Provides a stronger guarantee against negative backward transfer compared to regularization methods.

A rehearsal-based technique where a subset of data from past tasks is stored and interleaved with new task data during training. This helps the model rehearse old patterns while learning new ones.

• Key Mechanism: Maintains a replay buffer of historical data samples or their embeddings.
• Federated Challenge: Storing raw data on clients or server violates strict federated learning principles. Solutions include:
  • Storing synthetic data generated by a generative model.
  • Using federated coresets – small, representative subsets of data calculated via optimization.
• Use Case: Effective for tasks where generating representative synthetic data is feasible.

An architectural method that allocates new model parameters or entire sub-networks for each new task to avoid interference. Parameters for old tasks are typically frozen.

• Common Approaches:
  • Progressive Neural Networks: Add new columns (expert networks) for each task, with lateral connections to previous columns.
  • Adapter-based Methods: Add small, trainable adapter modules between frozen layers of a base model for each new task.
• Federated Consideration: Parameter growth must be communicated efficiently. Strategies include masking and sparse updates to transmit only the new, task-specific parameters.
• Benefit: Eliminates forgetting by design but leads to model size growth over time.

Trains a model's initialization (meta-learning) so that it can adapt quickly to a new task with few steps while being resilient to forgetting old tasks. Frameworks like Online-aware Meta-Learning (OML) are designed for sequential task arrival.

• Key Mechanism: The meta-training phase simulates a continual learning process across a distribution of tasks.
• Federated Adaptation: The meta-learning process itself can be conducted in a federated manner, learning an initialization that works well for rapid, decentralized adaptation across clients with non-stationary data.
• Use Case: Highly effective when tasks share a common structure and the goal is fast client-side adaptation.

Modifies the federated averaging (FedAvg) algorithm to be aware of task identity or data distribution shifts. Prevents the global model from drifting in a direction that harms performance on previous tasks.

• Techniques Include:
  • Multi-Model Aggregation: Maintaining separate global model parameters for different tasks or client clusters.
  • Gradient Filtering: Screening client updates for updates that conflict with the consolidated knowledge of past tasks before aggregation.
  • Adaptive Learning Rates: Applying different learning rates to parameters based on their estimated importance to past tasks.
• Core Principle: The server's aggregation logic is the primary defense against catastrophic forgetting in the global model.

Personalized Federated Learning is a paradigm focused on producing models tailored to the local data distribution of individual clients, rather than a single global model. It is closely related to Continual Federated Learning as personalization can help adapt to evolving local data streams.

• Objective: Balance shared global knowledge with client-specific adaptation.
• Common Techniques: Include training local heads on top of a shared feature extractor, using meta-learning to find easily adaptable initial parameters, or applying Multi-Task Learning frameworks.
• Synergy with Continual Learning: Personalization mechanisms can serve as a buffer against global catastrophic forgetting by allowing local models to retain task-specific knowledge.

Non-IID (Non-Independent and Identically Distributed) data is a fundamental characteristic of federated systems, where data across clients is statistically heterogeneous. This is a core challenge that Continual Federated Learning must address, as data distributions also shift over time.

• Manifestations: Variations in feature distribution (covariate shift), label distribution (prior probability shift), or concept distribution (concept shift) between clients.
• Consequences: Can cause client drift, slow convergence, and reduce final global model accuracy.
• Continual Aspect: Non-IIDness is compounded by temporal non-stationarity, where a single client's data distribution evolves, requiring techniques for both cross-client and cross-time adaptation.

Experience Replay is a biologically-inspired technique for continual learning where a model rehearses on a small, stored subset of data from previous tasks while learning new ones. In federated settings, this requires careful design due to privacy constraints.

• Mechanism: A replay buffer stores representative examples or embeddings. During training on new client data, these old examples are interleaved to remind the model of past knowledge.
• Federated Challenges: Storing raw data on clients or the server may violate privacy principles. Solutions include using generative replay (where a generative model creates synthetic old data) or storing only model outputs (e.g., logits) for distillation.
• Effectiveness: Highly effective at reducing forgetting but introduces memory overhead and algorithmic complexity.

Concept Drift Adaptation refers to methods that detect and respond to changes in the underlying statistical properties of a data stream over time. This is a critical component of Continual Federated Learning, as client data evolves.

• Types of Drift: Sudden drift (abrupt change), gradual drift, incremental drift, and recurring concepts.
• Detection Methods: Monitor performance metrics (accuracy, loss) or data distribution statistics (using KL-divergence, PCA) on client devices or the server.
• Adaptation Strategies: Upon detection, strategies include triggering a new learning round for affected clients, adjusting local learning rates, or re-weighting client contributions during aggregation to focus on those with current data.