Domain Generalization

An optimization framework that learns data representations which elicit the same optimal classifier across all training clients. IRM seeks to discover causal relationships robust to spurious correlations.

• Penalizes feature distributions that vary across domains
• Prevents reliance on hospital-specific shortcuts like scanner type
• Requires a gradient penalty term during federated aggregation
• Example: A pneumonia detector that learns lung opacity patterns rather than laterality markers from specific X-ray machines

A technique using a gradient reversal layer and domain discriminator to learn feature representations that are invariant to the client's domain. The model is trained to maximize domain confusion while minimizing task loss.

• Domain classifier tries to identify which hospital data came from
• Feature extractor learns to fool the domain classifier
• Results in site-agnostic embeddings
• Effective against feature distribution skew across different MRI vendors

A communication-efficient method where clients share compact class-representative vectors (prototypes) instead of full model updates. Each client computes the mean embedding for each class locally.

• Naturally handles label distribution skew
• Reduces communication overhead by orders of magnitude
• Prototypes are privacy-preserving aggregates, not raw data
• Example: Sharing the average embedding of 'malignant lesion' across dermatology clinics without sharing images

Techniques that explicitly minimize the distance between feature distributions of different clients using statistical measures. Common metrics include Maximum Mean Discrepancy (MMD) and CORAL loss.

• MMD measures distance in reproducing kernel Hilbert space
• CORAL aligns second-order statistics (covariance matrices)
• Applied as a regularization term during local training
• Reduces batch effects from different CT scanner protocols

A self-supervised approach that aligns representations of similar data instances across different clients while pushing apart dissimilar ones. Learns robust features without requiring labels.

• Positive pairs: same pathology from different hospitals
• Negative pairs: different pathologies regardless of source
• Uses InfoNCE loss to maximize mutual information
• Particularly effective when labeled data is scarce across the federation

Generates worst-case domain shifts during training to force the model to learn robust, generalizable features. Clients apply perturbations that maximize domain divergence.

• Creates synthetic variations of local data distributions
• Model learns to be invariant to extreme distributional shifts
• Combines with standard augmentation (rotation, color jitter)
• Prepares models for deployment at hospitals with radically different patient demographics

The fundamental challenge motivating domain generalization. In federated healthcare networks, local client datasets are not independent and identically distributed, reflecting natural variation across hospitals. This heterogeneity manifests as differences in patient demographics, disease prevalence, imaging equipment, and clinical protocols. A model that overfits to one hospital's distribution will fail catastrophically at another. Understanding the specific type of non-IIDness—whether label distribution skew, feature distribution skew, or concept drift—is the first step in selecting the appropriate generalization technique.

The measurable variation in data distributions across federated clients, encompassing differences in:

• Feature distributions P(x): Different patient demographics or scanner vendors
• Label distributions P(y): Varying disease prevalence rates
• Conditional distributions P(y|x): Different clinical diagnostic criteria

Domain generalization techniques must learn representations that are invariant to these distributional shifts. Unlike domain adaptation, which requires target domain data for alignment, generalization must work on entirely unseen distributions at deployment time.

A related but distinct paradigm from domain generalization. Domain adaptation assumes access to unlabeled target domain data during training, allowing the model to explicitly align feature distributions between source and target. Common techniques include Maximum Mean Discrepancy (MMD) minimization and adversarial domain confusion. In contrast, domain generalization must prepare for entirely unseen target domains, making it the more challenging and practically relevant problem for healthcare deployments where new hospital sites join the network without prior data access.

An optimization framework that learns data representations which elicit the same optimal classifier across all training clients. The core insight: if a feature representation leads to consistent predictions across diverse environments, it likely captures causal relationships rather than spurious correlations. The objective penalizes feature extractors where the optimal linear classifier varies significantly between clients. This approach is particularly valuable in healthcare, where models must avoid learning shortcuts like hospital-specific imaging artifacts instead of genuine pathological features.

A technique using a domain discriminator network with a gradient reversal layer to learn feature representations that are invariant to the client's domain. During training:

• The feature extractor tries to fool the domain classifier
• The domain classifier tries to identify which client the data came from
• The gradient reversal layer inverts gradients, pushing the feature extractor toward domain-agnostic representations

This adversarial game results in features that capture medically relevant information while suppressing site-specific biases, enabling robust generalization to new hospitals.