Federated Dataset Shift

The prior probability of class labels varies significantly across clients. This is the most common form of non-IIDness in clinical federated learning.

• Example: Hospital A is a specialized oncology center with 40% malignant cases, while Hospital B is a general clinic with only 5% malignant cases.
• Impact: A naive global model may overfit to the majority class of the largest client, leading to poor minority-class recall at other sites.
• Mitigation: Strategies include federated prototype learning, which shares compact class-representative vectors instead of full model updates, and federated knowledge distillation using a balanced public proxy dataset.

The marginal distribution of input features P(x) differs across clients, even if the conditional label distribution P(y|x) remains similar. This is often driven by demographic or equipment variations.

• Example: A pediatric hospital's chest X-rays have fundamentally different lung sizes and bone densities compared to a geriatric hospital's images.
• Example: MRI scans from a Siemens 3T scanner have different intensity histograms than those from a GE 1.5T scanner.
• Mitigation: Federated domain generalization via invariant risk minimization and federated feature alignment using Maximum Mean Discrepancy (MMD) loss.

A temporal form of dataset shift where the statistical relationship P(y|x) between input features and target labels changes over time. The definition of a disease itself evolves.

• Example: The clinical definition of sepsis (Sepsis-2 vs. Sepsis-3 criteria) changed, altering the ground-truth labels for the same patient vitals.
• Example: During the COVID-19 pandemic, the presentation of respiratory distress shifted, making pre-pandemic pneumonia classifiers unreliable.
• Mitigation: Federated drift detection monitors model performance and data distribution statistics across the network, triggering federated continual learning protocols to adapt without catastrophic forgetting.

A specific dataset shift where the distribution of input features P(x) changes between the federated training environment and the target deployment site, but the conditional label distribution P(y|x) remains constant.

• Example: A model trained on data from academic medical centers with high-resolution CT scanners is deployed to a rural clinic with older, low-dose scanners. The relationship between a lesion's appearance and malignancy is stable, but the input pixel distributions differ.
• Mitigation: Federated domain adaptation and federated harmonization techniques, such as removing scanner-specific batch effects, are critical.

A specific case of label distribution skew where only the distribution of the target variable P(y) changes, while the class-conditional feature distributions P(x|y) remain invariant across clients.

• Example: The radiographic presentation of a pneumothorax (collapsed lung) is a universal physiological fact, so P(x|y) is stable. However, a trauma center sees a much higher incidence of pneumothorax than an outpatient clinic, shifting P(y).
• Mitigation: This is often corrected by re-weighting the loss function during federated aggregation using estimates of each client's label prior, or through federated ensemble methods that combine local posterior probabilities.

While not a statistical shift, system heterogeneity is a core characteristic of the federated environment that exacerbates dataset shift. It refers to the variability in hardware, storage, and network capabilities across clients.

• Example: A large hospital trains on a GPU cluster and can participate in every round, while a small clinic on a single CPU may straggle or drop out, creating a biased view of the data distribution.
• Impact: Stragglers and dropped clients are often non-random, systematically excluding certain data distributions from the global model.
• Mitigation: Federated knowledge distillation allows clients with different model architectures to participate, and asynchronous aggregation protocols accommodate variable client speeds.