Model Personalization

A framework that trains personalized models for each client simultaneously by learning shared representations while allowing for client-specific model parameters to handle local data biases. Unlike standard FedAvg, which forces a single global model, multi-task learning explicitly models the task relationship between clients, treating each site's data distribution as a related but distinct learning problem. This is particularly effective when label distributions vary significantly across hospitals.

Applies knowledge gained from solving one problem in a federated network to a different but related problem. Critical when feature or label spaces differ across clients—for example, when one hospital collects genomic data while another relies on imaging. The approach typically involves:

• Feature extraction: Using a shared base model trained on common data modalities
• Domain adaptation: Aligning feature representations between source and target domains
• Fine-tuning: Adapting the transferred model on local client data without sharing raw patient records

Trains a model initialization across clients such that it can rapidly adapt to a new client's local data distribution with only a few gradient steps. Often implemented via algorithms like Reptile or MAML adapted for federated settings. The key insight: instead of finding a single model that works well on average, meta-learning finds parameters that are maximally adaptable. This is ideal for rare disease clinics where data is scarce but rapid personalization is essential.

Partitions clients into groups with similar data distributions and trains a separate model for each cluster. This mitigates the negative impact of extreme statistical heterogeneity without requiring fully individualized models. The clustering is typically performed based on:

• Gradient similarity: Clients whose local updates point in similar directions are grouped together
• Model weight cosine distance: Measuring the angular distance between client model parameters
• Performance-based clustering: Grouping clients where a shared model yields comparable accuracy

A privacy-preserving technique where clients share soft label predictions on a public dataset instead of model parameters. This enables:

• Heterogeneous model architectures: Each client can use a different neural network design
• Reduced communication costs: Logits are typically smaller than full model weights
• Enhanced privacy: Sharing predictions rather than gradients reduces vulnerability to model inversion attacks

The global model is trained to match the ensemble of local client predictions on the public reference dataset.

A communication-efficient method where clients share compact class-representative vectors (prototypes) instead of full model updates. Each prototype is the mean embedding vector for a given class computed on local data. The global server aggregates these prototypes to form a global class representation space. Local personalization occurs by regularizing each client's decision boundary toward the global prototypes while preserving local data geometry. This naturally handles label distribution skew without requiring all clients to have examples of every class.