FedProx

The defining innovation of FedProx is the addition of a proximal term to each client's local objective function. This term penalizes large deviations from the global model by adding (μ/2) * ||w - w_t||², where μ is a tunable hyperparameter controlling the penalty strength. This mechanism limits client drift by keeping local updates within a bounded region around the current global model, preventing aggressive overfitting to local Non-IID data distributions.

Unlike FedAvg, which requires all clients to complete a fixed number of epochs, FedProx introduces γ-inexactness to accommodate heterogeneous system resources. A client's local update is considered sufficient when the gradient norm falls below a threshold relative to the local objective: ||∇F_k(w)|| ≤ γ_k * ||∇F_k(w_t)||. This allows straggler clients with limited compute to contribute meaningful updates without completing full training, ensuring all participating hospitals can contribute regardless of infrastructure disparities.

FedProx provides theoretical convergence guarantees under statistical heterogeneity, a critical advantage for multi-institutional medical imaging. The framework provably converges even when client data is severely Non-IID—such as when one hospital specializes in pediatric radiology while another focuses on geriatric oncology. The proximal term acts as a stabilizer, preventing the global model from oscillating between conflicting local optima and ensuring consistent diagnostic performance across diverse patient populations.

FedProx decouples model convergence from uniform client participation. In real-world hospital networks, institutions may drop out mid-round due to network interruptions or clinical priorities. The framework's proximal term provides a correction mechanism that compensates for partial or intermittent participation, maintaining training stability even when only a subset of clients complete their local updates in any given communication round.

The proximal coefficient μ controls the trade-off between local adaptation and global consistency:

• μ = 0: Reduces to standard FedAvg with no regularization
• Small μ: Allows more local personalization, suitable for homogeneous client distributions
• Large μ: Enforces tighter global alignment, critical for highly heterogeneous Non-IID settings In medical imaging consortia, μ is typically tuned via cross-site validation to balance site-specific diagnostic accuracy with population-level generalizability.

FedProx provides rigorous convergence analysis for non-convex loss functions, which characterize deep neural networks used in medical image segmentation and classification. The framework guarantees convergence to a stationary point of the global objective under bounded gradient dissimilarity assumptions. This theoretical foundation is essential for FDA clearance pathways, where algorithmic stability and predictable behavior are regulatory prerequisites for Software as a Medical Device (SaMD).

The primary pathology FedProx is designed to correct. In standard Federated Averaging (FedAvg) , local models trained on Non-IID data diverge from the global optimum. This occurs because clients with heterogeneous data distributions pull the model in conflicting directions. FedProx's proximal term tethers local updates to the global model, explicitly limiting this divergence and stabilizing convergence.

The root cause of training instability in federated networks. It manifests as variations in data quantity, feature distribution, and label balance across hospitals. Unlike FedAvg, which assumes uniform local computation, FedProx provides a theoretical tolerance for this heterogeneity. It allows different clients to perform variable amounts of local work (γ-inexactness) without degrading the global model's accuracy.

The core mathematical innovation of FedProx. This is a regularization penalty added to the local objective function: min_w h_k(w; w^t) = F_k(w) + (μ/2) ||w - w^t||^2.

• μ (mu): A hyperparameter controlling the penalty strength.
• w^t: The current global model weights.
• Effect: It restricts local updates from straying too far from the global model, directly combating client drift in systems with partial work.

The baseline algorithm that FedProx generalizes. FedAvg performs SGD locally and aggregates via weighted averaging. Its critical vulnerability is Non-IID data: without a proximal term, heterogeneous clients cause the global model to diverge. FedProx reduces to FedAvg when the proximal coefficient μ is set to 0 and all clients complete uniform epochs.

The defining data challenge in cross-silo medical imaging. A hospital's dataset is not a uniform sample of the global population. Label distribution skew (one hospital has more positive cases) and feature distribution skew (different scanner vendors) are common. FedProx explicitly models this heterogeneity, providing convergence guarantees that FedAvg lacks in these realistic, skewed environments.

A mechanism allowing clients to solve their local subproblems imprecisely. In real-world federated systems, straggler devices or resource-constrained hospitals cannot always complete fixed epochs. FedProx introduces a γ-inexactness parameter that quantifies this tolerance, enabling the aggregation server to accept partial updates without sacrificing the stability of the global model.