pFedMe

pFedMe reformulates the federated optimization problem using Moreau envelopes, a mathematical tool from convex analysis. This decomposes the learning objective into two distinct subproblems:

• Global model learning: Solved at the server level to find a shared consensus
• Personalized model optimization: Solved locally by each client using a proximal term

The Moreau envelope smooths the local loss landscape, making the optimization more stable and allowing each client to find a personalized solution that is close but not identical to the global model.

The algorithm introduces an L2-norm regularization term between the personalized model and the global model. This proximal term acts as a mathematical tether:

• Prevents the local model from diverging too far from the global consensus
• Allows sufficient flexibility to adapt to non-IID local data distributions
• The regularization strength λ controls the personalization degree

A larger λ keeps models closer to the global consensus, while a smaller λ permits greater local adaptation. This provides a tunable knob for balancing generalization and personalization.

pFedMe employs a bi-level optimization structure that separates concerns cleanly:

• Inner loop: Each client solves its own personalized objective using the Moreau envelope, typically with multiple local update steps
• Outer loop: The server aggregates the personalized models to update the global model

This decoupling means clients can perform multiple local epochs without suffering from the client drift that plagues standard FedAvg. The inner optimization finds a true personalized solution rather than just taking a few gradient steps.

Standard FedAvg suffers from client drift when local data distributions are heterogeneous—each client's updates pull the global model in conflicting directions. pFedMe addresses this by:

• Allowing each client to optimize toward its own personalized objective rather than a shared one
• Using the Moreau envelope to maintain a controlled proximity to the global model
• Eliminating the tension between local adaptation and global consistency

This results in faster convergence and higher final accuracy, especially in highly non-IID settings common in healthcare data silos.

pFedMe provides rigorous convergence analysis under standard assumptions:

• Proves convergence for both strongly convex and non-convex loss functions
• Achieves a convergence rate of O(1/T) for strongly convex objectives
• Demonstrates that the Moreau envelope smoothing improves the conditioning of the local optimization problem

These theoretical foundations make pFedMe suitable for safety-critical healthcare applications where predictable convergence behavior is essential for regulatory approval and clinical validation.

By allowing clients to perform multiple local update steps within the inner optimization loop, pFedMe reduces the frequency of communication rounds:

• Clients solve their personalized subproblem more thoroughly before communicating
• Fewer communication rounds are needed to reach a target accuracy compared to FedAvg
• The global model update uses standard aggregation, maintaining compatibility with existing federated infrastructure

This is particularly valuable in healthcare settings where bandwidth is constrained or communication costs are high across institutional boundaries.

The mathematical foundation of pFedMe, a Moreau envelope is a smoothing technique from convex optimization. It decouples the personalized model optimization from the global model learning by creating a regularized subproblem for each client. This allows clients to pursue distinct local objectives without diverging destructively from the global consensus, effectively handling statistical heterogeneity in non-IID data distributions.

A precursor to pFedMe that explicitly optimizes the Model-Agnostic Meta-Learning (MAML) objective in a federated setting. Per-FedAvg finds a shared initial model from which clients can rapidly adapt to their local data with only a few gradient steps. Unlike pFedMe's proximal regularization, Per-FedAvg focuses on learning a sensitive initialization that maximizes personalization speed.

A personalization framework that adds a proximal term to the local objective function. This L2 regularization penalty encourages the local model to remain close to the global model while still optimizing for the client's specific empirical risk. Ditto's simplicity makes it a strong baseline, but pFedMe's Moreau envelope formulation provides a more theoretically principled separation of global and local objectives.

An alternative personalization strategy that partitions clients into groups with similar data distributions before performing standard Federated Averaging within each cluster. Techniques like Clustered Federated Learning recursively separate clients based on the similarity of their local model updates. This prevents divergent local objectives from degrading the global model, but requires explicit cluster identification unlike pFedMe's implicit personalization.

An algorithm that partitions the neural network into a shared global representation and a personalized local head. The base layers learn a common feature extractor across all clients, while the final classification layers remain unique to each client. This structural approach to personalization addresses data heterogeneity by allowing clients to maintain distinct decision boundaries on top of shared features.

A 'learning to learn' paradigm that trains a model initialization across clients such that it can rapidly adapt to a new local task with only a few gradient steps. Unlike pFedMe's per-round personalization, meta-learning optimizes for fast adaptation at deployment time. This approach is particularly valuable when new clients join the federation and need to personalize quickly with limited local data.