Agnostic Federated Learning (AFL)

AFL optimizes the global model for the worst-case mixture of client data distributions. Unlike Federated Averaging (FedAvg), which assumes a fixed target distribution, AFL minimizes the maximum expected loss over any possible mixture of the participating clients' distributions. This provides a minimax performance guarantee, ensuring the model does not catastrophically fail on any single node's data, even if that node's distribution is an outlier or adversarial.

AFL is inherently robust to Byzantine failures and adversarial data shifts. The framework does not assume clients are benign or identically distributed. By optimizing for the worst-case mixture, AFL mathematically limits the influence of any single malicious client attempting to skew the global model via data poisoning or submitting corrupted updates. This makes it a compelling alternative to Byzantine Fault Tolerance (BFT) aggregation rules like Krum or Trimmed Mean for non-IID environments.

The training process is formulated as a minimax game between the server and a hypothetical adversary:

• Server's Goal: Minimize the global model's loss.
• Adversary's Goal: Maximize the loss by choosing the worst-case weighting over client distributions. This game-theoretic setup converges to a Nash equilibrium, where the global model is optimally balanced across all potential client data shifts, preventing overfitting to any dominant distribution.

Key distinctions in optimization objectives:

• FedAvg: Minimizes the weighted average loss across clients, implicitly favoring clients with larger datasets. This leads to client drift and poor performance on minority distributions.
• AFL: Minimizes the maximum loss across any convex combination of client distributions. This explicitly prevents the model from ignoring small or statistically unique clinical sites.
• Convergence: AFL typically requires more communication rounds but achieves a Pareto-optimal solution that no single client can veto.

The core optimization problem is defined as: min_w max_λ Σ λ_k * L_k(w) where:

• w represents the global model parameters.
• λ is a probability vector over K clients, chosen by the adversary.
• L_k(w) is the empirical loss on client k's local data. This formulation directly addresses non-IID data handling by forcing the model to perform well on every possible weighting of the heterogeneous clinical silos.

AFL is critical for multi-institutional clinical networks where data distributions are inherently non-identical due to varying patient demographics, equipment vendors, and diagnostic coding practices. Use cases include:

• Rare Disease Detection: Guarantees the model does not ignore a small hospital specializing in a specific condition.
• Cross-Site Generalization: Prevents the model from overfitting to a large academic medical center's imaging protocols at the expense of community clinics.
• Regulatory Alignment: Supports Federated Regulatory Compliance by ensuring equitable model performance across all participating entities.