Federated Averaging (FedAvg)

A cryptographic protocol that encrypts individual client updates so the central server can only compute their weighted sum, never inspecting any single hospital's model contribution. This prevents gradient leakage attacks where an adversary could reconstruct patient data from raw model updates. SecAgg uses secret sharing and pairwise masking to ensure that even if the server is compromised, individual updates remain private. In healthcare consortia, this is often a regulatory prerequisite for FedAvg deployments.

The phenomenon where local models diverge from the optimal global model due to non-IID data distributions across hospitals. When a rural clinic's model overfits to its limited patient demographics while a research hospital trains on rare conditions, simple FedAvg averaging can produce a global model that performs poorly everywhere. Mitigation strategies include proximal regularization (FedProx), variance reduction (SCAFFOLD), and limiting local epochs to prevent excessive divergence before aggregation.

Standard FedAvg assumes all clients are honest, but in adversarial healthcare settings, a compromised node could send corrupted updates to poison the global model. Byzantine-resilient aggregation rules like Krum, Trimmed Mean, and Median replace simple weighted averaging with robust statistical operators that filter out outliers. These methods are critical when federating across institutions with varying security postures or when defending against model poisoning attacks targeting diagnostic models.

A mathematical framework that adds calibrated Gaussian noise to model updates before aggregation, providing a quantifiable privacy guarantee (ε, δ) that no single patient's data can be inferred. When combined with FedAvg, DP creates a DP-FedAvg variant where the central server clips and noisifies updates. Healthcare regulators increasingly require formal DP bounds, though practitioners must balance the privacy-utility tradeoff: higher noise improves privacy but degrades model accuracy on clinical tasks.

A quantitative metric measuring statistical heterogeneity across hospital datasets, typically computed using Earth Mover's Distance or Dirichlet distribution parameters. A high Non-IID Index warns that simple FedAvg may converge slowly or produce a biased global model. Common measurement approaches include:

• Label distribution skew: some hospitals have disproportionate disease prevalence
• Feature distribution skew: different imaging equipment produces varying pixel intensities
• Quantity skew: academic medical centers may have 100x more records than community clinics

An alternative to weight averaging where a global teacher model generates soft predictions (logits) on a public proxy dataset, and each hospital trains a local student model to match these predictions. Unlike FedAvg, this approach:

• Supports heterogeneous model architectures across institutions
• Transmits only prediction vectors, not model weights
• Reduces communication costs for large foundation models
• Enables personalized federated learning where local models adapt to site-specific patient populations while preserving global knowledge