Personalized Federated Learning

The foundational architectural pattern of PFL where the neural network is partitioned into shared global layers and personalized local layers. The global component learns universal feature representations across all clients, while the local component adapts to site-specific data distributions.

• FedPer keeps base layers synchronized globally while personalizing classification heads
• FedRep learns a shared representation with client-specific prediction heads
• Reduces communication overhead by only aggregating the global component
• Enables structural handling of feature distribution skew across clinical sites

A mathematical constraint that prevents local models from diverging too far from the global model during personalization. By adding a proximal term to the local objective function, algorithms balance adaptation to local data with the stability of the global consensus.

• FedProx adds an L2 penalty between local and global model weights
• Ditto extends this with a separate personalized model that trades off local empirical risk against global proximity
• Critical for non-IID clinical data where aggressive local fitting causes catastrophic forgetting
• The proximal coefficient λ controls the personalization-to-generalization ratio

A 'learning to learn' paradigm that trains a model initialization optimized for rapid local adaptation. Instead of finding a single global model that performs well everywhere, federated meta-learning finds a starting point from which clients can achieve high performance with minimal fine-tuning.

• Per-FedAvg explicitly optimizes the Model-Agnostic Meta-Learning (MAML) objective in a federated setting
• Enables few-shot personalization with only a handful of local gradient steps
• Reduces local compute burden compared to full retraining
• Particularly effective when new clinical sites join the federation with limited labeled data

A strategy that partitions clients into cohorts with similar data distributions before performing standard federated aggregation within each cluster. This prevents divergent local objectives from degrading model quality by maintaining multiple concurrent global models.

• Clustered Federated Learning (CFL) recursively separates clients based on gradient similarity
• Addresses multi-modal data distributions where a single global model is fundamentally insufficient
• Enables hospitals with similar patient demographics to share statistical strength without dilution from dissimilar sites
• Clustering can be based on model update similarity, data distribution metadata, or task relationship matrices

A communication-efficient personalization approach where clients share soft predictions (logits) on a public dataset rather than model weights. A student model learns from the ensemble of teacher models, transferring knowledge without exposing private model parameters.

• Federated Model Distillation enables heterogeneous model architectures across clients
• Clients can maintain completely different network topologies suited to their local compute resources
• Logit sharing provides stronger privacy guarantees than weight or gradient transmission
• Public proxy datasets must be carefully selected to represent the federation's domain

A lightweight personalization technique that creates client-specific models by mixing the parameters of a locally fine-tuned model and the globally aggregated model. The interpolation weight determines the balance between local specialization and global generalization.

• Simple convex combination: θ_personalized = α * θ_local + (1-α) * θ_global
• The mixing coefficient α can be optimized on a local validation set
• Requires no architectural changes to the base model
• Provides a computationally inexpensive baseline for personalization that often rivals more complex methods

Applies knowledge from a source domain to a target domain within a federated network. FTL addresses critical challenges in healthcare where labeled data is scarce and feature spaces are misaligned across institutions. For example, a model trained on radiology images at a large academic hospital can be adapted to a smaller community clinic with different scanner hardware. The technique uses domain-invariant feature extraction to bridge gaps between client data distributions without sharing patient records.

The process of further training a globally aggregated model on a specific client's local data post-federation. This step adapts model parameters to site-specific statistical nuances that the global model may have averaged away. Key considerations include:

• Learning rate decay to prevent overfitting on small local datasets
• Early stopping based on local validation performance
• Freezing early layers to preserve generalizable features while adapting later layers Local fine-tuning is often the simplest and most effective personalization strategy for clinical sites with sufficient local data volume.

An algorithm that partitions the neural network into a shared global representation and a personalized local head. The global body learns a common feature extractor across all clients, while each institution maintains a unique classifier tailored to its patient population. This structural separation addresses label distribution skew — a common scenario where different hospitals diagnose conditions at varying frequencies. FedRep reduces communication overhead by only aggregating the representation layers, leaving classification heads entirely local.

A technique that partitions clients into groups with similar data distributions before performing standard federated aggregation within each cluster. This prevents divergent local objectives from degrading model quality. Clustering approaches include:

• Gradient similarity: Grouping clients whose model updates point in similar directions
• Loss landscape analysis: Identifying clients with compatible optimization surfaces
• Demographic matching: Clustering hospitals by patient population characteristics Each cluster maintains its own global model, effectively creating multiple personalized models for distinct population subgroups.

A learning to learn approach that trains a model initialization across clients such that it can rapidly adapt to a new local task with only a few gradient steps. The objective explicitly optimizes for personalization speed rather than raw global accuracy. In clinical settings, this enables a new hospital joining the network to achieve high local performance after minimal fine-tuning. The Per-FedAvg algorithm implements this by finding a shared initial model that sits in a region of the loss landscape amenable to fast local adaptation.

A communication-efficient aggregation strategy where clients share class scores or logits on a public dataset instead of model weights. This transfers knowledge from a heterogeneous teacher ensemble to a student model without requiring architectural homogeneity. Benefits for healthcare personalization include:

• Heterogeneous model support: Each hospital can use its own preferred architecture
• Privacy amplification: Sharing soft labels rather than gradients reduces information leakage
• Natural personalization: The distilled student can be fine-tuned locally for site-specific performance