Domain Adaptation Attack

Adversaries inject poisoned training samples that corrupt the domain adaptation module's feature mapping function. The attack causes the model to learn an incorrect transformation between simulated feature distributions and real-world feature distributions. During deployment, the model systematically misclassifies real sensor inputs because the adaptation layer maps them to incorrect regions of the shared latent space. This is particularly dangerous because the model may perform perfectly in simulation validation while failing catastrophically in production.

Attackers manipulate the parameter distributions used during domain randomization training to create brittle policies. By subtly biasing the randomization ranges—such as narrowing lighting conditions or restricting texture variations—the adversary ensures the model only adapts successfully to a subset of real-world conditions. When deployed outside this narrow envelope, the agent's performance degrades sharply. This attack exploits the fact that domain randomization parameters are often treated as hyperparameters rather than security-critical configuration.

This sophisticated attack targets the gradient reversal layer commonly used in adversarial domain adaptation architectures. By crafting training batches where the domain classifier's gradients are deliberately misaligned, attackers cause the feature extractor to learn domain-variant features instead of domain-invariant ones. The model fails to generalize because it relies on simulation-specific artifacts—such as rendering noise patterns or physics engine quirks—that have no real-world correlate.

In CycleGAN-based domain adaptation, attackers exploit the cycle-consistency loss to inject hidden mappings. By introducing paired samples where the forward mapping (sim-to-real) and backward mapping (real-to-sim) are consistent but semantically incorrect, the adversary teaches the model to associate specific simulated features with wrong real-world categories. The attack survives validation because cycle-consistency metrics remain low while semantic accuracy collapses.

Attackers exploit the covariate shift assumption by introducing samples that violate the premise that label distributions remain consistent across domains. By poisoning the target domain's unlabeled data with examples where the conditional distribution P(Y|X) differs from the source domain, the adaptation module learns incorrect decision boundaries. This is especially effective in semi-supervised domain adaptation where target domain labels are unavailable for verification.

In domain adaptation methods using Wasserstein distance or optimal transport, attackers corrupt the cost matrix used to align source and target distributions. By inflating the transport cost between correct feature pairs and reducing it for incorrect ones, the adversary steers the Sinkhorn algorithm toward a deliberately wrong coupling. The resulting mapping preserves the mathematical properties of a valid transport plan while destroying semantic correspondence between domains.

Leverage unlabeled real-world data to enforce prediction stability. The model is penalized if its output for a real sample diverges significantly under different stochastic augmentations or dropout masks.

• Key Concept: Forces the adaptation module to learn a smooth manifold for the target domain, making it harder for poisoned data to create sharp, exploitable decision boundaries.
• Technique: Apply FixMatch or Mean Teacher paradigms specifically to the domain adaptation bridge.
• Outcome: The model becomes resistant to small, adversarial shifts in the target feature distribution.

Architect the encoder to explicitly separate domain-specific features (lighting, texture) from domain-agnostic semantic features (object shape, structure).

• Defense Logic: The domain adaptation attack targets the domain-specific channel. By isolating it, the semantic content stream remains uncompromised.
• Architecture: Use a Variational Autoencoder (VAE) framework with a domain classifier that operates only on the isolated domain latent code.
• Result: Even if the domain mapping is corrupted, the agent's core understanding of the scene remains intact.

Continuously monitor the output distribution of the domain adaptation module during inference using a CUSUM (Cumulative Sum) or SPRT (Sequential Probability Ratio Test) detector.

• Metrics Tracked: Maximum Mean Discrepancy (MMD) between incoming feature batches and a trusted baseline distribution.
• Response: If a drift is detected, the system can fall back to a simulation-only policy or request human intervention.
• Advantage: Provides a last line of defense that operates without needing labels in the target domain.