Non-IID Data

The marginal probability distribution P(y) of labels differs across clients. A hospital specializing in cardiology will have a disproportionate number of cardiac cases compared to a general practice clinic.

• Example: Hospital A has 40% pneumonia cases and 5% pneumothorax; Hospital B has 5% pneumonia and 30% pneumothorax.
• Impact: Local models overfit to dominant local classes, causing client drift during federated aggregation.
• Mitigation: FedProx adds a proximal term to constrain local updates from diverging too far from the global model.

The marginal probability distribution P(x) of input features varies across clients. Different hospitals use different scanner manufacturers, acquisition protocols, and slice thicknesses.

• Example: Hospital A uses Siemens scanners with 1mm slices; Hospital B uses GE scanners with 3mm slices.
• Impact: A model trained on one scanner's texture patterns fails to generalize to another's, even for the same pathology.
• Mitigation: Domain adaptation and histogram matching normalize feature distributions before training.

The conditional probability P(y|x) differs across clients—the same image features map to different diagnoses at different institutions due to varying diagnostic criteria or population prevalence.

• Example: A chest X-ray with subtle interstitial markings may be labeled 'normal' at a community hospital but 'interstitial lung disease' at a tertiary referral center.
• Impact: The global model receives contradictory gradient signals, leading to confused decision boundaries.
• Mitigation: Personalized federated learning allows each client to maintain a locally adapted model head while sharing feature extractor layers.

The number of training samples varies dramatically across clients, creating an unbalanced contribution to the global model during aggregation.

• Example: A large academic medical center contributes 500,000 annotated scans while a rural clinic contributes 2,000.
• Impact: Naive FedAvg weighting by local dataset size causes the global model to be dominated by large institutions, erasing rare patterns from smaller sites.
• Mitigation: Weighted aggregation with capped influence or federated distillation using a public reference dataset balances contributions.

Data distributions at each client evolve over time due to changing equipment, updated clinical guidelines, or shifting patient populations—a phenomenon distinct from static Non-IIDness.

• Example: A hospital upgrades from CT to photon-counting CT mid-study, fundamentally altering image texture statistics.
• Impact: A model trained on historical data becomes progressively stale, with accuracy degrading as the gap between training and inference distributions widens.
• Mitigation: Continuous federated learning with drift detection triggers selective retraining rounds when distribution metrics exceed thresholds.

Not all clients collect the same data modalities. Some hospitals have paired CT and PET scans, while others have only CT, creating structural missingness in the feature space.

• Example: Hospital A provides CT + pathology reports; Hospital B provides CT only.
• Impact: Standard multi-modal fusion architectures fail when expected input channels are absent at inference time.
• Mitigation: Modality-agnostic encoders and dropout-based training simulate missing modalities during federated rounds to build robustness.