Partial Model Personalization

The neural network is explicitly divided into shared global layers and personalized local layers. Typically, base feature extractors (early convolutional or transformer blocks) remain globally synchronized to learn universal representations, while classification heads or final layers are personalized. This partitioning is not arbitrary—it exploits the empirical finding that lower layers capture general features (edges, textures, syntax) while higher layers encode task-specific semantics.

FedPer formalizes the layer-partitioning approach by keeping base layers synchronized across all clients via standard federated averaging while allowing each client to maintain unique personalized top layers. The key insight: heterogeneous data distributions primarily affect decision boundaries, not feature extraction. By personalizing only the classification layers, FedPer reduces the parameter divergence problem that plagues fully personalized models while maintaining strong local performance on non-IID clinical data.

FedRep introduces a two-phase local training procedure: first, the shared representation network is updated using all clients' data; second, each client optimizes its personalized prediction head independently. This decoupling prevents the gradient interference that occurs when global and local objectives compete. The representation learns domain-invariant features while each head specializes to its site's label distribution—critical for hospitals with different diagnostic coding practices.

By transmitting only shared layer updates and keeping personalized parameters local, partial personalization dramatically reduces communication costs compared to full model personalization. For a ResNet-50 with a personalized final block, the communication payload drops by 60-80% versus transmitting all parameters. This makes the approach viable for cross-silo healthcare deployments where bandwidth between hospitals may be constrained or where frequent synchronization is required for regulatory compliance.

Partial personalization inherently resists catastrophic forgetting because the globally synchronized layers act as a stability anchor. When a hospital fine-tunes on new local patient data, the shared feature extractor preserves knowledge from the broader federation. This contrasts with full local fine-tuning, where aggressive adaptation can overwrite globally learned representations. The shared layers function as an implicit elastic weight consolidation mechanism without requiring explicit importance-weight calculations.

Partial personalization directly addresses non-IID data distributions across clinical sites. When hospitals serve demographically distinct populations—pediatric vs. geriatric, urban vs. rural—their local label distributions diverge. By personalizing decision boundary layers while sharing feature extraction, the model accommodates label distribution skew without forcing a single global classifier to compromise. This yields 15-25% accuracy improvements on local test sets compared to purely global models in highly heterogeneous federations.

A foundational architecture that partitions a neural network into shared base layers and personalized classification heads. The global server aggregates only the base layers, which learn a universal feature representation, while each client retains a private, locally-trained classifier. This directly addresses structural data heterogeneity where clients share the same feature space but have different label distributions. The key insight is that lower layers capture generalizable patterns, while upper layers encode task-specific semantics.

An algorithm that formalizes the distinction between global representation and local head learning. Unlike FedPer, FedRep alternates between training the shared feature extractor and the personalized classifier in distinct phases. Clients first optimize their local heads for several epochs, then freeze them to compute gradients for the global representation. This prevents the local objective from distorting the shared feature space, leading to superior performance on non-IID data with significant label skew.

The simplest personalization strategy: after the server distributes a globally aggregated model, each client performs additional training epochs on its own local dataset. This adapts all or selected parameters to site-specific statistical nuances without further communication. While computationally lightweight, it risks catastrophic forgetting of global knowledge, especially when local datasets are small. Often combined with elastic weight consolidation or proximal regularization to constrain deviation from the global optimum.

A post-hoc personalization technique that constructs a client-specific model by mixing the parameters of the global model and a fully local model. The interpolation weight is optimized on a local validation set to find the optimal balance point. Formally: θ_client = α * θ_local + (1-α) * θ_global. This provides a smooth spectrum between complete personalization and complete federation, allowing each client to independently determine its ideal trade-off based on local data volume and distribution shift.

A personalization framework that adds a proximal term to the local objective function, explicitly penalizing the L2 distance between the local model and the most recent global model. The regularized loss becomes: L_local + (λ/2) * ||θ_local - θ_global||². This encourages local models to remain in the vicinity of the global solution while still adapting to local data. The hyperparameter λ controls the personalization-globalization trade-off, with higher values enforcing stronger global consistency.

A 'learning to learn' paradigm that trains a model initialization across clients such that it can rapidly adapt to any client's local data with only a few gradient steps. Algorithms like Per-FedAvg explicitly optimize the Model-Agnostic Meta-Learning (MAML) objective in a federated setting. The server distributes a meta-model; each client computes a Hessian-vector product to simulate fine-tuning, then sends meta-gradients back. This is ideal for few-shot personalization where clients have very limited labeled data.