Adversarial Domain Randomization

Unlike standard domain randomization, which uniformly samples parameters to maximize robustness, ADR frames the attacker as an adversarial agent that actively searches for parameter configurations that minimize policy performance. The attacker learns to propose hard negative examples—specific combinations of friction, mass, lighting, or sensor noise—that cause catastrophic failure. This is formalized as a minimax game where the adversary maximizes the policy's loss while the policy minimizes it, leading to a brittle equilibrium.

The attacker injects a biased sampling distribution into the training loop, replacing the intended uniform or Gaussian randomization ranges. Key mechanisms include:

• Range narrowing: Constraining randomization to a narrow band where the policy overfits, leaving it unable to generalize outside that band.
• Distribution shifting: Moving the mean of a parameter distribution to an unrealistic value, causing the policy to learn incorrect dynamics.
• Correlation injection: Introducing false correlations between independent parameters (e.g., linking high friction with low mass) that never co-occur in reality.

The primary outcome of ADR is a policy that exhibits high variance in the real world despite low variance in simulation. The attacker exploits the sim-to-real gap by ensuring the policy's failure modes align precisely with conditions present in the deployment environment. For example, a robotic grasping policy might be trained with adversarial randomization that excludes the specific coefficient of friction of the target object, causing consistent slippage that was never observed during evaluation.

Defending against ADR requires continuous distributional auditing of the randomization engine. Security teams should:

• Monitor the Kullback-Leibler divergence between the intended and actual parameter sampling distributions.
• Implement two-sample statistical tests (e.g., Kolmogorov-Smirnov) to detect unauthorized distribution shifts.
• Track per-episode parameter histograms to identify anomalous clustering that indicates adversarial manipulation.
• Use cryptographic provenance to verify that randomization seeds and ranges have not been tampered with in the training pipeline.

ADR shares deep conceptual roots with reward function hacking and specification gaming. In both cases, the attacker exploits the gap between the training objective and the true deployment objective. While reward hacking manipulates the reward signal directly, ADR manipulates the environment distribution to achieve the same effect: a policy that scores well on training metrics but fails on the true task. Sophisticated attacks combine both techniques, using ADR to create environmental conditions where reward hacking is easier to execute.

A robust defense involves inverting the adversarial dynamic by explicitly training against an ADR adversary. This is implemented as a three-player game:

• Policy: Learns to maximize task reward.
• Adversary: Learns to propose challenging parameter configurations.
• Validator: Learns to detect when the adversary is proposing unrealistic or out-of-distribution parameters. The validator acts as a realism constraint, ensuring the adversary cannot drift into physically impossible parameter regimes, forcing it to find genuinely challenging but realistic scenarios.

An adversary with knowledge of the training pipeline identifies that lighting intensity is randomized uniformly between 0.1 and 1.0 during simulation. By deploying the vehicle exclusively in environments with lighting at exactly 0.05 intensity—just outside the training distribution—the perception system fails to detect obstacles. The policy, never exposed to this edge case, exhibits a brittle failure mode where confidence scores collapse and the vehicle stops making decisions entirely.

A warehouse robot's grasping policy is trained with coefficient of friction randomized between 0.3 and 0.9. An attacker introduces a targeted surface treatment to specific packages that yields a friction coefficient of exactly 0.25. The policy, which learned to rely on friction as a stable feature within its training range, applies insufficient grip force and drops the package. This demonstrates how adversarial parameter selection exploits the boundaries of randomization ranges.

A delivery drone's stabilization policy is trained with wind gust frequency randomized between 0.5 Hz and 5.0 Hz. An attacker deploys a directed acoustic or mechanical device that generates precisely 6.0 Hz oscillations—a frequency the policy never encountered. The out-of-distribution input causes the controller to enter a resonant amplification loop, resulting in catastrophic loss of stability. The randomization range itself becomes the attack vector.

A visual inspection system is trained with camera exposure time randomized between 1ms and 20ms. An attacker installs a strobe light synchronized to the inspection cycle, forcing effective exposure to 0.5ms. The resulting underexposed frames fall outside the model's learned feature space, causing defect detection to fail silently. The system reports normal operation while missing critical quality defects, demonstrating how sensor parameter attacks bypass randomization-based robustness.

A legged robot's locomotion policy is trained with ground compliance randomized across a range representing soil, gravel, and asphalt. An attacker prepares a surface with compliance precisely at 0.02—softer than any training condition. The policy's learned impedance control parameters overcompensate, causing the robot to sink and become immobilized. This reveals how randomization without adversarial boundary testing creates policies vulnerable to parameter extrapolation.

An autonomous underwater vehicle's navigation policy is trained with water density randomized to simulate fresh, brackish, and salt water. An attacker identifies that the randomization excludes the halocline transition zone where density changes rapidly over small depth intervals. By guiding the vehicle into an estuary with sharp salinity gradients, the buoyancy control system oscillates violently, exhausting battery reserves. The attack exploits the gap between discrete randomization points and continuous real-world variation.