Personalized Federated Learning

Frameworks like Per-FedAvg leverage meta-learning to find an initial global model that can be rapidly adapted to a new client's local data distribution with only a few gradient steps. The goal shifts from finding a single global minimizer to finding a shared initialization that is sensitive to local fine-tuning.

• Mechanism: The global model is trained explicitly on the ability to adapt, not just on aggregate accuracy.
• Benefit: Dramatically reduces the number of local training epochs required for a new client to achieve high diagnostic accuracy on their specific patient demographic.

A class of PFL techniques that learns an optimal mixture between the global model and a purely local model. The mixing weight is often a learned function of the local data distribution, allowing clients with highly divergent data to rely more on their local training.

• Implementation: The final personalized model is a convex combination: θ_personalized = α * θ_local + (1 - α) * θ_global.
• Use Case: A rural clinic with a unique, skewed patient demographic can automatically weight its local model higher than the global consensus to prevent underfitting on its specific population.

Treats each client's local optimization as a distinct but related task within a multi-task learning framework. Instead of forcing all clients toward a single weight vector, the system learns a shared representation while allowing client-specific model parameters to diverge in a structured, regularized manner.

• Key Insight: Enforces relationships between client models using a task covariance matrix or Laplacian regularization.
• Outcome: Models for hospitals with similar imaging protocols will be structurally closer in parameter space, while outliers are not penalized for diverging.

A PFL strategy that recursively partitions clients into hierarchical clusters based on the geometric similarity of their local model updates or loss landscapes. A separate global model is then trained for each cluster, effectively grouping clients with similar data distributions.

• Algorithm: Iterative bi-partitioning using cosine similarity of gradient updates.
• Advantage: Prevents a hospital with advanced MRI scanners from degrading the model performance of a partner using older CT technology by isolating them into distinct optimization groups.

Architectures that explicitly separate a neural network into shared base layers (capturing universal features) and personalized head layers (capturing client-specific biases). Only the base layers are aggregated globally, while the head remains local.

• Structure: Base layers learn general anatomical features; personalized heads learn site-specific labeling preferences or scanner calibration biases.
• Efficiency: Reduces communication overhead since only a fraction of the model parameters are transmitted to the aggregation server.

Instead of aggregating weights, clients exchange soft predictions (logits) on a public, unlabeled reference dataset. Each client then distills the collective knowledge into its own personalized model architecture, which can be completely heterogeneous.

• Privacy: No model weights are shared, only low-dimensional prediction vectors, providing an additional layer of security against model inversion attacks.
• Flexibility: Allows each hospital to maintain a completely custom model architecture optimized for its own hardware and inference constraints.