Domain Adaptation

The core mechanism involves learning a transformation that maps source domain data and target domain data into a shared feature space where their distributions are indistinguishable. This is achieved by minimizing a domain discrepancy metric such as Maximum Mean Discrepancy (MMD) or Correlation Alignment (CORAL).

• Adversarial methods use a domain discriminator with a gradient reversal layer to force the feature extractor to produce domain-invariant representations.
• Statistical methods explicitly match first-order (mean) and second-order (covariance) statistics between domains.
• The goal is to retain discriminative power for the primary task while discarding domain-specific information.

The most practical and widely studied setting where the model has access to labeled source data but only unlabeled target data. This mirrors real-world clinical deployments where annotating data at every new hospital is prohibitively expensive.

• Techniques leverage pseudo-labeling: generating confident predictions on target data and using them as supervisory signal.
• Self-training iteratively refines pseudo-labels to reduce confirmation bias.
• Entropy minimization encourages the model to make high-confidence predictions on target samples, pushing decision boundaries away from dense data regions.
• UDA is essential for scaling diagnostic models across institutions without requiring manual annotation at each site.

A seminal architecture that introduces a gradient reversal layer (GRL) between the feature extractor and a domain classifier. During forward propagation, the GRL acts as an identity transform; during backpropagation, it multiplies the gradient by a negative constant.

• The feature extractor is trained to maximize domain classifier loss (fooling the discriminator).
• The domain classifier is trained to correctly identify which domain a feature vector originates from.
• This minimax game converges when features become domain-invariant.
• DANN is particularly effective for feature distribution skew where input characteristics vary across sites but the diagnostic task remains consistent.

Combines domain adaptation with privacy-preserving decentralized learning. Instead of centralizing data from source and target domains, adaptation occurs through shared model parameters or intermediate representations.

• Federated adversarial training extends DANN by placing a domain discriminator at each client or the server.
• Federated feature alignment uses MMD or CORAL loss computed on local client batches and aggregated at the server.
• Federated prototype alignment shares compact class-representative vectors across clients to align feature spaces without exposing raw data.
• This approach is critical for healthcare federated learning where label distribution skew and feature distribution skew coexist across hospitals.

A paradigm where adaptation occurs during inference on the target domain without access to source data. The model updates its parameters online using only the current target batch.

• Batch normalization adaptation replaces source statistics with target batch statistics during inference.
• Entropy minimization fine-tunes model parameters to reduce prediction uncertainty on each incoming batch.
• Self-supervised auxiliary tasks like rotation prediction provide a proxy training signal on unlabeled target data.
• TTA is ideal for edge deployment scenarios where a pre-trained model must adapt to local patient demographics on a medical device without any cloud connectivity.

Extends the adaptation framework to scenarios where labeled data exists from multiple distinct source domains, each with its own distribution. The goal is to generalize to an unseen target domain by leveraging the diversity of sources.

• Domain aggregation learns a single model that aligns all source domains simultaneously.
• Mixture-of-experts dynamically weights source-specific feature extractors based on target sample characteristics.
• Moment matching aligns the moments of feature distributions across all source-target pairs.
• This setting is highly relevant for federated learning where each hospital represents a distinct source domain, and the combined knowledge must transfer to a new participating institution.