Federated Continual Learning

Each client maintains a small, privacy-compliant memory buffer of representative samples from previous tasks. During local training on new data, these stored exemplars are interleaved to regularize weight updates and prevent catastrophic forgetting.

• Pseudo-rehearsal: Generative models can synthesize proxy samples instead of storing raw data
• Privacy compliance: Buffers are stored locally and never shared with the server
• Memory constraints: Typical buffers use only 1-5% of local storage capacity

The model architecture dynamically allocates task-specific sub-networks or masks to prevent gradient interference between sequentially learned distributions. When a new clinical task emerges, only a sparse subset of parameters is updated.

• Hard attention masks: Binary gates freeze parameters critical to prior tasks
• Progressive network expansion: New capacity is added laterally without modifying existing weights
• Overlap control: Shared representations are carefully managed to balance forward transfer and retention

Instead of aggregating raw weights, clients exchange soft targets or logits computed on a globally accessible unlabeled dataset. This model-agnostic approach allows heterogeneous local architectures while preserving task knowledge.

• Server-side distillation: The global model is trained to match the ensemble of client teacher outputs
• Co-distillation: Peers exchange predictions directly in decentralized topologies
• Temporal ensembling: Historical predictions are accumulated to stabilize learning across tasks

A Fisher information matrix is computed after each task to identify parameters critical for performance. Subsequent training applies a quadratic penalty proportional to parameter importance, anchoring essential weights in place.

• Online Laplace approximation: Fisher diagonals are updated incrementally without storing full matrices
• Federated Fisher merging: Clients share Fisher information alongside model updates for global consolidation
• Synaptic intelligence: Alternative importance measures track each parameter's contribution to loss reduction over trajectories

The central server maintains separate aggregation buffers for each learned task. When new client updates arrive, they are routed to the appropriate task-specific model head while shared feature extractors are updated with elastic constraints.

• Multi-head architectures: Task-specific output layers prevent logit interference
• Task boundary detection: Drift detection algorithms automatically identify distribution shifts without explicit task IDs
• Class-incremental protocols: New diagnostic categories are added without retraining on historical patient data

During local optimization, gradients computed on new data are projected into a subspace orthogonal to gradients from stored memory samples. This constrained optimization ensures that updates to the model do not increase loss on previously learned tasks.

• Projected gradient descent: Quadratic programming solves for minimal interference updates
• Efficient approximations: Random projection techniques reduce computational overhead
• Federated gradient alignment: The server can enforce global orthogonality constraints across client subspaces

The primary challenge addressed by continual learning, where a neural network abruptly loses performance on previously learned tasks upon training on new data. In a federated context, this is exacerbated by non-IID data distributions across hospitals. Mitigation strategies include elastic weight consolidation (EWC), which penalizes changes to parameters critical for prior tasks, and experience replay, which interleaves synthetic or buffered samples from older distributions during local training.

A defining characteristic of real-world clinical networks where local datasets are not independently and identically distributed. This statistical heterogeneity causes local model divergence during sequential learning. Continual learning algorithms must handle label distribution skew (different disease prevalence per site) and concept drift (changing diagnostic criteria over time) without violating the data locality principle.

A synaptic consolidation algorithm that computes a Fisher information matrix to identify parameters crucial for previous tasks. During federated continual learning, the global server can aggregate these importance matrices to guide local training. This acts as a quadratic penalty in the loss function, anchoring critical weights and enabling the model to learn new disease phenotypes without erasing prior diagnostic capabilities.

Techniques to mitigate forgetting by replaying representative data from prior distributions. In privacy-preserving healthcare settings, this often uses generative replay—training a local generative adversarial network (GAN) or variational autoencoder (VAE) to produce synthetic samples of old tasks. The central server can also curate a federated coreset of anonymized, aggregated prototypes shared across the network.

A continual learning strategy where the model's capacity grows over time by adding new parameters or modules for each sequential task. In a federated setting, this requires model heterogeneity management—the global server must orchestrate a growing, partially shared architecture. Techniques like progressive neural networks freeze old columns and add lateral connections, preventing interference while enabling forward transfer of clinical knowledge.

The statistical monitoring of data distribution shifts over time within each federated client. Continual learning systems must distinguish between virtual drift (changes in the input distribution P(X)) and real drift (changes in the conditional distribution P(Y|X)). Federated drift detection algorithms aggregate local KL-divergence or MMD metrics to trigger model adaptation rounds without centralizing raw patient data.