Statistical Heterogeneity

Occurs when the prior probability of class labels P(y) varies significantly across clinical sites. A specialized oncology center may have 40% malignant cases while a general hospital sees only 5%. This imbalance causes naive federated averaging to bias the global model toward majority-class clients.

• Example: Hospital A (cardiology specialty): 60% abnormal, 40% normal ECGs; Hospital B (general): 10% abnormal, 90% normal
• Impact: Global model overfits to Hospital A's abnormal patterns
• Mitigation: FedProx, FedNova, or class-balanced aggregation weights

The marginal distribution of input features P(x) differs across clients even when labels are consistent. In medical imaging, this arises from different scanner manufacturers (Siemens vs. GE), acquisition protocols, or patient demographics.

• Example: Site A scans with 1.5T MRI (lower resolution), Site B uses 3T MRI (higher resolution)
• Impact: Model learns spurious correlations tied to imaging artifacts rather than pathology
• Mitigation: Federated domain adaptation, feature alignment with MMD, or federated batch normalization

The conditional distribution P(x|y) varies across clients — the same disease presents with different feature patterns at different sites. This reflects genuine clinical variation in patient populations, comorbidities, or diagnostic criteria.

• Example: Pneumonia presents with different radiographic patterns in elderly vs. pediatric populations
• Impact: A model trained predominantly on adult data fails on pediatric cases
• Mitigation: Federated multi-task learning, clustered federated learning, or personalized model heads

Clients contribute vastly different volumes of training data, from large academic medical centers with millions of records to rural clinics with only hundreds. This violates the assumption that each client's update carries equal statistical weight.

• Example: Major hospital: 500,000 samples; Community clinic: 2,000 samples
• Impact: Large clients dominate the global model; small clients' unique patterns are diluted
• Mitigation: Weighted aggregation proportional to dataset size, FedAvg with proximal terms, or knowledge distillation from small clients

Data distributions evolve over time within the same client due to changing clinical practices, updated diagnostic guidelines, or seasonal disease patterns. A model trained on pre-pandemic chest X-rays may fail on post-pandemic data with different disease prevalence.

• Example: COVID-19 emergence shifts the distribution of respiratory conditions in emergency departments
• Impact: Static global models become progressively less accurate without continuous adaptation
• Mitigation: Federated continual learning, drift detection monitors, and periodic global model retraining

The pattern of missingness itself is non-IID across clinical sites. Some hospitals routinely collect genetic markers while others lack sequencing capabilities. Missingness may be informative (not random), introducing systematic bias.

• Example: Tertiary care centers have complete lab panels; community clinics lack specialized biomarker tests
• Impact: Global model learns to rely on features unavailable at resource-limited sites
• Mitigation: Federated data imputation, feature selection based on availability, or modality-agnostic architectures

The foundational condition where local client datasets are not independent and identically distributed. In healthcare, this is the norm—not the exception—as patient populations, disease prevalence, and clinical protocols vary inherently across institutions. Non-IIDness is the root cause of statistical heterogeneity and manifests in three primary forms:

• Label Distribution Skew: Hospitals specialize in different conditions
• Feature Distribution Skew: Different patient demographics or scanner vendors
• Concept Drift: Evolving diagnostic criteria over time

A type of non-IID data where the prior probability of class labels varies significantly across clients. A tertiary referral center may have a 40% prevalence of rare cancers, while a community hospital sees only 2%. This imbalance causes naive federated averaging to bias the global model toward the majority classes of dominant clients.

• Extreme case: Some clients may have zero examples of certain classes
• Mitigation: Federated prototype learning or class-balanced aggregation weights

Occurs when the marginal distribution of input features differs across clients, even if label relationships remain consistent. In medical imaging, this is driven by:

• Scanner variability: Siemens vs. GE MRI machines produce different intensity distributions
• Demographic drift: A pediatric hospital vs. a geriatric care center
• Protocol differences: Contrast-enhanced vs. non-contrast CT acquisition Feature skew breaks the IID assumption of standard SGD, causing local optima to diverge.

A temporal form of non-IIDness where the statistical relationship between features and labels changes over time. Unlike static skew, concept drift requires continuous adaptation:

• Real-world example: The clinical definition of sepsis was updated (Sepsis-3 criteria in 2016), changing the ground truth labels for the same physiological features
• COVID-19 impact: Pneumonia classification models trained pre-2020 faced sudden concept drift as novel viral patterns emerged
• Detection: Federated drift detection monitors monitor performance degradation across sites

The capability of a single global model to perform accurately on entirely unseen client sites without any local adaptation. Unlike domain adaptation, which requires target data, domain generalization learns representations invariant to spurious site-specific features.

• Key techniques: Federated invariant risk minimization (IRM) and federated adversarial training with gradient reversal layers
• Clinical value: A model trained on 10 hospitals should work immediately at an 11th hospital with a different scanner vendor and patient demographic

A strategy that partitions clients into groups with similar data distributions and trains a separate model for each cluster. This avoids forcing a single global model to compromise between fundamentally incompatible data distributions.

• Clustering signals: Model update similarity, feature distribution distance, or metadata (hospital type, region)
• Hierarchical approach: A multi-tier architecture where cluster-level models are optionally aggregated into a meta-model
• Trade-off: Improved per-cluster accuracy at the cost of reduced generalization to entirely new clusters