Bayesian Aggregation (FedBE)

Instead of a simple weighted average, the server fits a Gaussian or Dirichlet distribution to the incoming local model updates. This captures the epistemic uncertainty arising from client heterogeneity. The global model is then sampled from this high-probability region, ensuring the aggregated result is statistically consistent with all client contributions rather than being an arbitrary mean.

FedBE generates the final global model by drawing Monte Carlo samples from the inferred posterior distribution and averaging them. Key advantages:

• Smooths out outlier client updates without explicit Byzantine detection
• Naturally handles multi-modal weight distributions caused by non-IID clinical data
• Produces a global model that lies in a high-density region of the parameter space

Standard FedAvg often produces overconfident predictions on out-of-distribution patient data. FedBE's Bayesian treatment directly addresses this. By marginalizing over the posterior, the global model's predictive confidence aligns with its true empirical accuracy. This is critical for clinical decision support where a model must know when it is uncertain about a rare pathology.

In non-IID settings, local models drift toward different minima. A point estimate average can land in a low-probability valley between these modes. FedBE's posterior explicitly models this multi-modality. Sampling from the posterior and averaging the samples effectively performs Bayesian model averaging, yielding a solution that generalizes better to the aggregate data distribution than any single point.

FedBE provides a principled mechanism for decomposing uncertainty:

• Aleatoric uncertainty: Inherent noise in the clinical data
• Epistemic uncertainty: Model uncertainty due to limited or heterogeneous training data This decomposition allows a hospital to flag predictions with high epistemic uncertainty for mandatory clinician review, creating a safety net for rare or ambiguous cases.

FedBE is an aggregation-agnostic wrapper that can be applied on top of existing federated frameworks. The local training procedure remains unchanged. The server simply replaces the deterministic averaging step with:

1. Fit a variational approximation to the client updates
2. Draw K samples from the posterior
3. Average the samples to produce the global model This modularity allows easy integration into existing FedAvg or FedOpt pipelines.

The foundational aggregation algorithm that constructs a global model by computing a weighted average of locally trained model updates. Unlike FedBE, which models a full posterior distribution, FedAvg computes a point estimate of the global weights.

• Weights are proportional to local dataset sizes
• Assumes IID data distributions across clients
• Serves as the baseline against which Bayesian methods are compared

A model fusion technique that aggregates local knowledge by matching the logit outputs or soft labels of a global student model to the ensemble of local teacher models. FedBE shares a conceptual link here: both methods leverage distributional information rather than raw weight averaging.

• Transfers dark knowledge between models
• Agnostic to model architecture heterogeneity
• Complements Bayesian approaches for uncertainty calibration

Treats locally trained models as an ensemble and combines their predictions via voting or averaging at inference time. FedBE's Bayesian approach can be viewed as a principled method for constructing a posterior-weighted ensemble.

• No single global model is distilled
• Naturally captures predictive diversity
• FedBE formalizes this with Bayesian model averaging

A class of robust aggregation rules designed to ensure convergence even when a subset of nodes submits arbitrary or malicious updates. Bayesian aggregation provides a natural framework for BFT by down-weighting outliers based on their likelihood under the posterior.

• Krum and Trimmed Mean are deterministic alternatives
• FedBE offers probabilistic outlier rejection
• Critical for cross-silo healthcare deployments with untrusted sites

An aggregation mechanism that injects calibrated statistical noise into the model update process to provide formal privacy guarantees. Bayesian aggregation pairs naturally with DP: the posterior distribution already encodes uncertainty, and additional noise can be interpreted through a Bayesian lens.

• Provides (ε, δ)-differential privacy guarantees
• FedBE can incorporate noise into the prior or likelihood
• Trade-off between privacy budget and posterior precision

Uses Moreau envelopes to decouple personalized model optimization from global model learning. FedBE's Bayesian framework extends personalization by maintaining a distribution over global parameters, allowing each client to sample a personalized model from the posterior.

• Handles statistical heterogeneity across sites
• Bayesian personalization via posterior sampling
• Relevant for site-specific clinical model adaptation