FedProx vs FedAvg for Heterogeneous Clients

FedAvg (Federated Averaging) excels at efficiency in ideal, homogeneous networks because its core assumption is that client data is Independent and Identically Distributed (IID). For example, in controlled simulations with uniform client compute, FedAvg achieves fast convergence with minimal communication rounds, making it the established baseline for federated learning systems like TensorFlow Federated (TFF) and Flower (Flwr).

FedProx takes a different approach by introducing a proximal term to the local client objective function. This strategy explicitly handles statistical (non-IID) and systems (straggler) heterogeneity by restricting local updates to stay closer to the global model. This results in a trade-off: improved stability and fairness across diverse clients at the cost of slightly increased per-round computation and potential convergence deceleration in perfectly homogeneous settings.

The key trade-off: If your priority is raw speed and simplicity in near-IID, reliable networks, choose FedAvg. If you prioritize robust convergence and fairness in real-world, heterogeneous environments with variable client capabilities—common in healthcare (HIPAA) or finance (GDPR) cross-silo collaborations—choose FedProx. For a deeper dive into algorithmic robustness, see our comparison of Byzantine-Robust Federated Learning vs FedAvg.

FedProx excels at handling both statistical (non-IID) and systems (straggler) heterogeneity because it introduces a proximal term to the local client objective. This term acts as a regularizer, penalizing local updates that stray too far from the global model, which stabilizes training and improves convergence in imbalanced environments. For example, in a benchmark with 100 clients exhibiting high data skew, FedProx has been shown to reduce the number of communication rounds to reach target accuracy by up to 30% compared to FedAvg, while also tolerating a wider range of local epochs per client.

FedAvg takes a different approach by performing simple weighted averaging of client model updates. This results in a highly efficient and simple-to-implement baseline but is vulnerable to client drift when local data distributions diverge significantly. The key trade-off is speed versus stability: FedAvg can converge faster in ideal, near-IID conditions with uniform client participation, but its performance degrades sharply under the heterogeneous conditions common in real-world deployments like healthcare or finance.

The key trade-off: If your priority is robust convergence and tolerance for stragglers in a production environment with inherent client variability, choose FedProx. Its proximal term provides the necessary guardrails. If you prioritize maximal simplicity and speed in a controlled, simulated environment or where client data is relatively homogeneous, the classic FedAvg remains a valid starting point. For deeper insights into managing heterogeneity, explore our guide on Byzantine-Robust Federated Learning vs FedAvg and the architectural implications in Cross-Silo vs Cross-Device Federated Learning.