Federated Proximal Aggregation (FedDyn)

FedDyn modifies the local objective function by adding a linear penalty term derived from the historical gradient of the global model. Unlike FedProx, which uses a static quadratic penalty, this dynamic regularizer adapts over communication rounds to actively nudge local optima toward the global stationary point. The penalty term is updated on the server after each aggregation round, creating a feedback loop that progressively aligns local and global objectives.

The core innovation of FedDyn is its ability to eliminate client drift without requiring additional communication rounds. In standard FedAvg, heterogeneous local data causes each client's model to converge to a different local minimum. FedDyn's dynamic regularizer mathematically guarantees that, upon convergence, all local stationary points coincide with the global stationary point. This is achieved by maintaining a persistent state variable that accumulates the discrepancy between local and global gradients.

FedDyn provides theoretical convergence guarantees for both convex and non-convex loss functions. The algorithm achieves a convergence rate of O(1/T) for strongly convex objectives and converges to first-order stationary points for non-convex settings. These guarantees hold under standard assumptions of smoothness and bounded variance, making FedDyn one of the few federated algorithms with formal proofs of exact convergence in heterogeneous environments.

Unlike stateless aggregation methods, FedDyn maintains a persistent global gradient estimate on the server across communication rounds. This state variable acts as a memory of the global optimization trajectory, enabling the server to apply momentum-based optimizers like SGD with momentum or Adam during aggregation. The stateful design allows FedDyn to leverage advanced optimization techniques typically reserved for centralized training.

FedDyn achieves communication efficiency by reducing the total number of rounds required for convergence. Because the dynamic regularizer actively corrects for client drift, each local training epoch contributes more effectively toward the global optimum. Empirical studies show FedDyn can reduce communication rounds by 30-50% compared to FedAvg and FedProx on non-IID data partitions, without requiring gradient compression or sparsification techniques.

FedDyn is inherently robust to partial client participation, a critical requirement for cross-device federated learning where only a fraction of clients are available in each round. The persistent global gradient state compensates for missing client updates, preventing the global model from being biased toward the subset of participating clients. This property makes FedDyn suitable for large-scale deployments with unreliable or intermittently available edge devices.

FedDyn excels at training diagnostic models across radiology departments with heterogeneous scanner vendors and non-IID patient demographics. The dynamic regularization penalty ensures that a hospital with predominantly geriatric chest X-rays does not pull the global model away from a pediatric hospital's data distribution.

• Converges to a shared stationary point despite vendor-specific pixel intensity variations
• Eliminates the performance degradation seen with FedAvg on feature shift across sites
• Maintains diagnostic accuracy for rare pathologies present at only one institution

Intensive care units exhibit extreme label distribution skew—a trauma center sees vastly different mortality patterns than a cardiac specialty hospital. FedDyn's penalty term, based on the historical gradient of the global model, prevents local models from overfitting to their site-specific outcome distributions.

• The linear penalty term dynamically adjusts to keep local objectives aligned
• Avoids the catastrophic forgetting of minority outcomes common in FedAvg
• Enables robust early warning scores that generalize across diverse ICU populations

Electronic health record phenotyping suffers from systematic coding biases—one health system may use ICD-10 codes differently than another. FedDyn's convergence guarantees ensure that the global phenotype classifier settles at a true consensus rather than oscillating between conflicting local optima.

• The dynamic regularizer acts as a convergence anchor for inconsistent labeling practices
• Outperforms FedProx when local data volumes vary by orders of magnitude
• Produces a single global model that satisfies the stationary conditions of all participating sites

For rare disease detection, individual hospitals possess extremely small positive cohorts—sometimes fewer than 50 confirmed cases. FedDyn's formulation explicitly solves for a global model that is a consensus of all local stationary points, preventing larger hospitals from dominating the optimization landscape.

• The dual variable tracks the long-term gradient mismatch for each client
• Prevents the global model from ignoring rare disease signals from small sites
• Provides theoretical convergence guarantees even with severely unbalanced local datasets

Digital pathology suffers from stain normalization drift—different labs use different hematoxylin and eosin staining protocols, creating systematic color and texture variations. FedDyn's proximal mechanism ensures that local models do not overfit to lab-specific staining artifacts while still learning diagnostically relevant features.

• The global stationary point represents stain-invariant histological features
• Eliminates the need for manual stain normalization preprocessing
• Maintains consistent grading accuracy across labs with different slide preparation workflows

Pharmaceutical trials require consistent patient stratification across geographically dispersed sites with varying electronic health record implementations. FedDyn's convergence to a shared stationary point ensures that a patient classified as 'high-risk' at one site receives the same classification at another.

• The historical gradient penalty prevents site-specific drift in risk thresholds
• Enables regulatory-compliant federated model validation without data centralization
• Supports adaptive trial designs where enrollment criteria evolve based on global model consensus

The foundational aggregation algorithm that constructs a global model by computing a weighted average of locally trained model updates. Unlike FedDyn, FedAvg does not include a dynamic regularizer to align local and global stationary points, making it more susceptible to client drift in non-IID settings.

• Mechanism: Server averages client weights proportionally to dataset size
• Weakness: Struggles with statistical heterogeneity
• FedDyn Advantage: FedDyn adds a penalty term to explicitly correct for objective inconsistency

A federated optimization framework that adds a proximal term to local objective functions to stabilize convergence. While FedProx limits local updates from diverging too far from the global model, FedDyn goes further by using a dynamic regularizer based on the historical gradient of the global model to actively align stationary points.

• Proximal term: μ/2 ||w - w_t||² penalty added locally
• FedDyn distinction: FedDyn's regularizer evolves over rounds; FedProx uses a static μ
• Use case: Both handle systems heterogeneity, but FedDyn provides stronger theoretical convergence guarantees

A stochastic controlled averaging algorithm that corrects for client drift using control variates. SCAFFOLD maintains a server-side control variate (c) and client-side variates (c_i) to estimate the gradient direction of the global objective. FedDyn achieves a similar drift-correction effect through its gradient penalty term without requiring clients to store and transmit separate control variates.

• Control variates: Track the difference between local and global gradient directions
• Communication: SCAFFOLD transmits both model updates and control variates (2x communication per round)
• FedDyn efficiency: Achieves comparable convergence with only model parameter transmission

A cryptographic protocol that allows the server to compute the sum of encrypted client updates without inspecting individual contributions. SecAgg is complementary to FedDyn—while FedDyn defines what mathematical operation to perform on the updates, SecAgg ensures how that operation is executed without exposing private gradients.

• Technique: Secret sharing and pairwise masking
• Integration: FedDyn's proximal term computation can be performed over SecAgg-encrypted updates
• Privacy guarantee: Server learns only the aggregated model, never individual local gradients

A framework using Moreau envelopes to decouple personalized model optimization from global model learning. Like FedDyn, pFedMe addresses statistical heterogeneity, but focuses on producing client-specific models rather than a single global consensus. FedDyn's dynamic regularizer naturally allows local models to retain beneficial personalization while still converging globally.

• Moreau envelope: A smoothed approximation enabling bi-level optimization
• Personalization vs. consensus: pFedMe optimizes for local performance; FedDyn balances both
• Synergy: FedDyn's penalty term can be adapted to incorporate pFedMe-style personalization objectives

Robust aggregation rules ensuring convergence even when some nodes submit arbitrary or malicious updates. FedDyn's proximal term provides an implicit form of robustness by anchoring local updates to the historical global gradient, but explicit BFT mechanisms like Krum or Trimmed Mean can be layered on top of FedDyn for defense against active adversaries.

• Threat model: Up to f out of n clients may be Byzantine
• Krum: Selects the update with minimal sum of squared distances to neighbors
• FedDyn + BFT: Apply robust aggregation to the proximal-regularized updates for dual protection