Federated Averaging (FedAvg)

The core mechanism of FedAvg is a weighted linear combination of model parameters. The global model update is computed as:

• Formula: w_global = Σ (n_k / n) * w_k
• Mechanism: Each client's contribution is scaled by the size of its local dataset (n_k) relative to the total data (n).
• Impact: Prevents a small client with anomalous data from dominating the global model, ensuring statistical stability.

FedAvg decouples model training from the central server by performing multiple local optimization steps.

• Process: Instead of sending raw gradients, clients compute full weight updates by running several epochs of SGD on their private data.
• Hyperparameter: The number of local epochs (E) is a critical tuning parameter.
• Trade-off: Increasing E reduces communication overhead but risks client drift if local data is highly Non-IID.

FedAvg is designed to minimize the bottleneck of network transfer.

• Reduced Rounds: By computing significant local updates, FedAvg requires fewer communication rounds to reach convergence compared to naive Federated SGD.
• Payload: Only model weights (or weight deltas) are transmitted, not raw training data.
• Optimization: Techniques like gradient compression and quantization can be layered on top of FedAvg to further reduce bandwidth requirements.

The algorithm introduces a stochastic client selection parameter (C) to manage scale.

• Mechanism: In each round, the server randomly samples a fraction C of the total available clients to participate.
• Scalability: This allows the system to scale to millions of devices without requiring all nodes to be online simultaneously.
• Statistical Impact: Random selection introduces noise but provides an unbiased estimate of the full population gradient.

A fundamental challenge for standard FedAvg is statistical heterogeneity.

• Problem: When client data is Non-IID (e.g., different hospitals have different disease prevalences), local objectives diverge from the global optimum.
• Symptom: This causes client drift, where the averaged global model performs worse than a centrally trained model.
• Mitigation: Extensions like FedProx add a proximal term to the local loss to anchor updates to the global model.

FedAvg provides a foundation for privacy, but does not guarantee it alone.

• Gradient Leakage: Raw weight updates can still be analyzed via model inversion attacks to reconstruct private data.
• Defense Stacking: FedAvg is typically combined with Secure Aggregation (to hide individual updates from the server) and Differential Privacy (to add calibrated noise to updates).
• Compliance: This layered approach is essential for meeting HIPAA and GDPR requirements in medical imaging.

The primary challenge FedAvg must overcome in medical imaging. Local datasets at different hospitals are not independently and identically distributed—one hospital may have predominantly geriatric chest X-rays while another specializes in pediatric trauma.

• Causes client drift where local models diverge from the global optimum
• Reflects real-world demographic and equipment variations
• FedAvg's simple averaging can struggle with extreme statistical heterogeneity
• Mitigated by variants like FedProx that add proximal regularization terms

A cryptographic protocol that pairs with FedAvg to ensure the central server can only compute the sum of encrypted model updates without inspecting any individual hospital's weight contributions in plaintext.

• Uses secret sharing to mask individual updates
• Server learns only the aggregated average, never individual gradients
• Critical for HIPAA compliance in multi-hospital diagnostic model training
• Prevents gradient leakage attacks that could reconstruct patient scan features

A single complete cycle in federated training consisting of three phases:

1. Distribution: Server broadcasts the current global model to selected client hospitals
2. Local Training: Each hospital trains on its private DICOM data for a configurable number of epochs
3. Aggregation: Server applies FedAvg to weight updates and produces a new global model

The number of communication rounds is a critical hyperparameter balancing convergence quality against network bandwidth costs in clinical environments.

The divergence of locally trained models from the optimal global objective caused by heterogeneous, Non-IID data distributions across participating hospitals.

• Manifests when local SGD steps pull weights toward local optima
• FedAvg's periodic synchronization partially corrects drift
• Excessive drift degrades global model diagnostic accuracy
• Addressed by FedProx and SCAFFOLD algorithms that introduce correction terms
• Particularly problematic in rare disease detection where only a few sites have positive cases

The deployment topology where FedAvg is most commonly applied in healthcare. Designed for a small, reliable number of institutional participants (typically 2-50 hospitals) rather than millions of mobile devices.

• Each silo holds a large, curated dataset with substantial compute resources
• Clients are stateful and participate in most or all training rounds
• Assumes honest-but-curious participants rather than malicious actors
• Enables collaborative training of diagnostic models across hospital networks without centralizing protected health information

A mathematical framework often layered on top of FedAvg to provide provable privacy guarantees. Injects calibrated noise into model updates before aggregation.

• Controlled by the privacy budget (epsilon)—lower values enforce stronger guarantees
• Protects against membership inference and model inversion attacks
• Introduces a utility-privacy tradeoff: excessive noise degrades diagnostic accuracy
• Can be applied at the client level (local DP) or server level (central DP)
• Essential for satisfying regulatory requirements under GDPR and HIPAA