Federated Ensemble Aggregation (FedEnsemble)

The foundational aggregation algorithm that constructs a global model by computing a weighted average of locally trained model updates. Each client's contribution is typically weighted by its local dataset size.

• Mechanism: Server initializes a global model, distributes it to clients, clients train locally for multiple epochs, and the server averages the resulting model weights.
• Key Insight: Multiple local epochs reduce communication rounds compared to FedSGD, but can cause client drift on non-IID data.
• Limitation: Struggles with heterogeneous data distributions and varying client computational capabilities.

A model fusion technique that aggregates local knowledge by matching the logit outputs or soft labels of a global student model to the ensemble of local teacher models, rather than averaging weights.

• Advantage: Enables aggregation of heterogeneous model architectures across clients, unlike weight-averaging methods that require identical model structures.
• Process: Local models act as teachers, generating soft predictions on a public or synthetic dataset; the global student learns from these aggregated predictions.
• Privacy Benefit: Only model outputs are shared, not model weights or gradients, providing an additional layer of abstraction.

A class of robust aggregation rules designed to ensure the global model converges correctly even when a subset of participating nodes submits arbitrary, malicious, or corrupted updates.

• Threat Model: Assumes up to f out of n clients may be Byzantine adversaries attempting to poison the global model.
• Defense Strategies: Includes Krum (selecting the most consensus-aligned update), Trimmed Mean (discarding extreme coordinate values), and Median aggregation.
• Healthcare Relevance: Critical for cross-institutional networks where a compromised hospital could inject adversarial updates to degrade diagnostic model performance.

A cryptographic protocol that allows a central server to compute the sum of encrypted client model updates without being able to inspect any individual contribution, ensuring data privacy.

• Mechanism: Clients encrypt their updates using pairwise masking and secret sharing; the server can only decrypt the aggregate sum, not individual vectors.
• Threat Model: Protects against an honest-but-curious server that follows the protocol but attempts to infer private training data from individual updates.
• Integration: Often combined with differential privacy to provide defense-in-depth against both server-side and model-output privacy attacks.

An aggregation mechanism that injects calibrated statistical noise into the model update process, providing a formal mathematical guarantee that individual patient data cannot be inferred from the global model.

• Gaussian Mechanism: Adds noise proportional to the sensitivity of the update and inversely proportional to the privacy budget ε.
• Privacy Budget (ε): A tunable parameter controlling the privacy-utility tradeoff; lower ε provides stronger privacy but degrades model accuracy.
• Clinical Application: Enables compliance with HIPAA and GDPR by providing provable bounds on information leakage from trained diagnostic models.

A federated optimization framework that enhances Federated Averaging by adding a proximal term to local objective functions, stabilizing convergence across heterogeneous clinical devices and non-IID data distributions.

• Proximal Term: Penalizes local model updates that deviate too far from the current global model, controlled by a hyperparameter μ.
• Partial Work Handling: Accommodates straggler clients by allowing variable amounts of local computation rather than requiring all clients to complete a fixed number of epochs.
• Theoretical Guarantee: Provides convergence bounds even under systems heterogeneity where FedAvg may diverge.