Model Interpolation

The core operation is a convex combination of parameter vectors. For a client (k), the personalized model parameters (\theta_k^{\text{personalized}}) are computed as:

(\theta_k^{\text{personalized}} = \alpha \cdot \theta^{\text{global}} + (1 - \alpha) \cdot \theta_k^{\text{local}})

• (\theta^{\text{global}}): The parameters of the model aggregated from all participating clients.
• (\theta_k^{\text{local}}): The parameters of a model trained exclusively on client (k)'s private data.
• Interpolation Coefficient ((\alpha)): A scalar hyperparameter, typically (0 \leq \alpha \leq 1), that controls the blend. An (\alpha) of 1.0 yields the pure global model, while 0.0 yields the pure local model.

The interpolation coefficient (\alpha) is the critical control knob. Its value is determined by:

• Data Similarity: A higher (\alpha) (more global) is used when local data distribution is similar to the global distribution.
• Local Data Volume: Clients with scarce data benefit from a higher (\alpha) to avoid overfitting.
• Task Specificity: Highly user-specific tasks (e.g., next-word prediction) use a lower (\alpha).

Adaptive Alpha: Advanced implementations dynamically adjust (\alpha) per client or per round based on local validation loss, effectively performing a form of client-centric aggregation.

Model interpolation typically follows a strict two-stage pipeline:

1. Federated Stage: A global model (\theta^{\text{global}}) is trained collaboratively using a standard algorithm like Federated Averaging (FedAvg).
2. Local Interpolation Stage: After federation concludes, each client:
   • Trains a model (\theta_k^{\text{local}}) from scratch (or fine-tunes the global model) on its private dataset.
   • Computes the weighted average: (\theta_k^{\text{personalized}} = \alpha \cdot \theta^{\text{global}} + (1 - \alpha) \cdot \theta_k^{\text{local}}).

This decouples collaborative learning from personalization, simplifying the federation process.

Interpolation directly manages the classic bias-variance trade-off in the federated, non-IID context.

• Global Model (High (\alpha)): Lower variance, but potentially high bias for a specific client's data distribution. It represents the consensus knowledge.
• Local Model (Low (\alpha)): Lower bias for the local task, but high variance and risk of overfitting due to limited data.

By interpolating, the personalized model seeks an optimal point, reducing overfitting while retaining task-specific relevance. This is a form of hypothesis transfer learning with an explicit regularization path.

Interpolation can be applied to entire models or specific components. Layer-wise personalization is a common refinement:

• Global Layers: Early feature extractors (e.g., convolutional layers in a CNN) are often interpolated with a high (\alpha) to retain general representations.
• Personalized Layers: Final classification or regression heads may use a very low or zero (\alpha), effectively becoming pure personalized model heads. This is architecturally similar to FedPer or FedRep but with a smooth blending mechanism. This allows different parts of the network to retain different amounts of global versus local knowledge.

Model interpolation is a foundational concept with clear relationships to adjacent methods:

• Local Fine-Tuning: Can be seen as a special case where (\theta_k^{\text{local}}) is initialized from (\theta^{\text{global}}) and (\alpha) is implicitly annealed during training.
• Personalized Federated Learning with Regularization: Adding an L2 penalty (|\theta_k - \theta^{\text{global}}|^2) to the local loss approximates the effect of interpolation, pulling the solution toward a weighted average.
• Multi-Task Federated Learning: Interpolation produces a client-specific model that is a linear combination of two extreme tasks: the global consensus task and the purely local task.
• Knowledge Distillation for PFL: The global model can act as a teacher for the local model before interpolation, refining the local component.

Model interpolation directly mitigates the challenges of Non-IID (Non-Independent and Identically Distributed) data, which is the norm in federated settings. By blending a global model (trained on aggregated, diverse data) with a locally fine-tuned model, it creates a personalized model that balances general knowledge with client-specific patterns.

• Example: In next-word prediction for mobile keyboards, the global model knows common language patterns, while the local model learns a user's unique slang and phrasing. Interpolation produces a model that is both generally competent and personally relevant.

This technique enhances privacy by keeping raw personal data on-device. The sensitive local model, trained solely on private data, never leaves the client. Only its parameters (or the interpolated model) might be shared, which is a less direct exposure than raw data.

• It operates within the Privacy-Preserving Machine Learning paradigm, complementing techniques like secure aggregation.
• The local model acts as a privacy-preserving representation of the user's data distribution, which is then combined with federated knowledge.

Model interpolation can reduce system-wide resource consumption compared to training fully personalized models from scratch.

• Compute Efficiency: Clients often perform fewer local training epochs, as they start from a strong global initialization and only need to adapt it.
• Communication Efficiency: After the initial global model is distributed, subsequent communication can be limited to transmitting only the interpolation weight or the final interpolated model, rather than multiple rounds of gradient updates. This is a key concern in Communication-Efficient Federated Learning.

Interpolation acts as a regularizer against catastrophic forgetting, where a model overwrites previously learned, generally useful knowledge when trained on new, specific data.

• The global model anchors the personalized model to the shared feature space.
• The interpolation weight controls the trade-off: a high weight on the global model preserves shared knowledge; a high weight on the local model maximizes personalization. This is a form of Personalized Federated Learning with Regularization.

It enables systems to adapt to client drift—changes in a client's local data distribution over time. The interpolation weight can be dynamically adjusted.

• Example: A health monitoring model for a user whose activity patterns change seasonally. The server can provide updated global models (trained on population trends), while the local model continuously adapts. The interpolation mechanism seamlessly blends these streams for a model that is both current and personalized.

Strategic interpolation can help mitigate biases that may exist in either the global or local models.

• A global model may reflect majority population biases.
• A local model may overfit to a small, potentially biased local dataset.
• Interpolation allows a system designer to tune the influence of each component, potentially creating a fairer model for the client than either extreme. This intersects with goals in Federated Model Evaluation Metrics for fairness.

A foundational PFL algorithm that extends standard Federated Averaging (FedAvg). Instead of forcing all clients to converge to a single global model, pFedAvg allows for client-specific model parameters. The server aggregates only a subset of layers, or clients perform local fine-tuning after receiving the global aggregate, creating a personalized model for each participant. It is the conceptual precursor to more formal interpolation methods.

A straightforward PFL technique where personalization is a distinct, sequential phase. Each client receives the latest global federated model from the server and then performs additional training epochs exclusively on its local dataset. This adapts the general model to the client's specific data distribution. Model interpolation can be viewed as a more integrated approach, blending the global and locally fine-tuned models throughout the training process.

An architectural PFL strategy where a global set of specialized sub-models (experts) is learned collaboratively. A client-specific gating network, trained locally, determines how to weight or select these experts for inference. The final personalized model is a dynamic interpolation of the global expert bank, conditioned on the client's data. This differs from direct parameter interpolation by using a learned routing function.

Frames personalization as a transfer learning problem. The global model serves as the source hypothesis. During local adaptation, the client's model is regularized to stay close to this source, preventing catastrophic forgetting of useful shared knowledge while adapting to local data. Model interpolation is an explicit, often linear, instantiation of this idea, where the personalized model is a convex combination of the source (global) and target (local) hypotheses.

A class of optimization algorithms designed for the PFL objective. These methods aim to find a set of personalized model parameters efficiently. They may involve:

• Client-specific gradient corrections
• Biased local updates steered toward a personalized optimum
• Adaptive server aggregation (FedOpt) Model interpolation defines the form of the solution (a weighted average), while personalized optimization defines the process to find it.

A technique addressing feature distribution shift across clients, a key challenge for interpolation. FedBN keeps batch normalization layer parameters (mean and variance statistics) local to each client and does not aggregate them. This allows the same global convolutional or dense layers to produce effective features for all clients, even with non-IID data, creating a more stable foundation for subsequent interpolation or personalization steps.