FedProx

The core innovation of FedProx is the addition of a proximal term to each client's local objective function. This term penalizes large deviations of the local model from the global model during training, effectively limiting the impact of client drift. The proximal coefficient μ controls the trade-off between local adaptation and global consistency. A larger μ keeps local updates closer to the server model, which is critical when training on highly heterogeneous clinical datasets where some sites may have rare disease presentations not represented elsewhere.

FedProx introduces a γ-inexactness criterion that allows clients to perform variable amounts of local computation. Unlike FedAvg, which mandates a fixed number of local epochs, FedProx permits clients to solve their local subproblems approximately. A client satisfies γ-inexactness if the gradient norm of its local objective is bounded by γ times the gradient norm at the starting point. This accommodates straggler devices in clinical IoT networks—such as older MRI machines or underpowered edge servers—that cannot complete full local training within a synchronization window.

FedProx provides theoretical convergence guarantees under non-IID data distributions that violate the assumptions of standard FedAvg. The proximal term acts as a stabilizer when local data distributions diverge significantly—common in healthcare where different hospitals serve distinct demographic populations. The framework bounds the dissimilarity between local and global objectives, ensuring that even clients with highly skewed label distributions (e.g., a rural clinic with predominantly geriatric cases) contribute meaningfully to the global model without destabilizing convergence.

FedProx explicitly models systems heterogeneity—the variability in compute, memory, and network resources across participating nodes. By decoupling local computation from a rigid epoch count, the framework allows high-resource nodes to perform more precise local updates while resource-constrained devices contribute approximate solutions. This is essential for cross-device federated learning in healthcare, where training may span GPU clusters at academic medical centers and CPU-only inference chips on wearable monitors, all participating in the same collaborative training round.

In real-world clinical deployments, not all nodes are available for every training round due to network outages, maintenance windows, or operational constraints. FedProx provides convergence guarantees under partial participation, where only a random subset of clients contributes per round. The proximal term ensures that the global model does not overfit to the active subset, maintaining stability even when critical data sources are intermittently unavailable. This is vital for hospital networks where emergency department systems may drop offline during peak load.

The proximal coefficient μ is a critical hyperparameter that controls the strength of regularization toward the global model:

• μ = 0: Reduces exactly to standard FedAvg, with no proximal constraint
• Small μ: Allows significant local adaptation, suitable for homogeneous client populations
• Large μ: Enforces tight global consistency, ideal for highly heterogeneous or adversarial settings This tunability makes FedProx adaptable across diverse clinical deployment scenarios, from tightly controlled research networks to loosely coupled regional health information exchanges.