Domain Randomization

This principle involves randomizing the visual appearance of the simulation to prevent the policy from overfitting to specific textures, colors, or lighting conditions. By varying these elements, the model learns to focus on the geometric and semantic structure of objects and scenes.

• Examples: Randomizing object textures (wood, metal, plastic), floor colors, wall patterns, and global lighting (intensity, direction, color temperature).
• Goal: Achieve appearance invariance, ensuring a robot can recognize a 'cup' regardless of its visual style.

This principle randomizes the physical properties and simulation dynamics to bridge the reality gap in how objects move and interact. The policy learns robust control under varying physical conditions.

• Examples: Randomizing object mass, friction coefficients, motor torque limits, actuator latency, and gravity. For a grasping policy, this might involve varying an object's weight and surface slipperiness.
• Goal: Achieve dynamics robustness, so a policy trained to push an object works whether the surface is icy or carpeted.

The core learning objective of Domain Randomization is to force the neural network's latent representations to become invariant to superficial domain shifts. The model is incentivized to discard non-essential features (like a specific shadow pattern) and retain essential ones (like object shape).

• Mechanism: By making correlated visual/dynamics features unpredictable across training episodes, the only reliable signal for reward is the underlying task-relevant state.
• Outcome: The policy develops a generalized understanding that transfers to the unseen dynamics and visuals of the real world.

Effective Domain Randomization requires a strategic balance. Random variation (e.g., random colors) provides broad coverage, while systematic variation targets known, critical sim-to-real gaps.

• Systematic Example: If a real camera has a specific lens distortion, systematically simulating a range of similar distortions is more effective than purely random blur.
• Best Practice: Start with broad randomizations (textures, lighting) and layer in targeted, systematic variations for known hardware discrepancies (e.g., specific actuator delay ranges measured on the real robot).

This principle reframes the reality gap—the difference between simulation and reality—not as a single point but as a distribution of possible realities. Domain Randomization trains a policy to perform well across this entire distribution.

• Philosophy: Instead of perfecting one simulation (which is impossible), create a family of simulations whose combined variability encompasses the real world.
• Result: The real world becomes just another unseen instance from the broad training distribution, enabling robust zero-shot transfer.

An advanced extension where the range and parameters of randomization are automatically tuned during training. ADR starts with a narrow distribution and progressively expands it in directions where the policy is failing, creating a curriculum of increasing difficulty.

• Process: The system identifies easy and hard domains within the current randomization space, then expands the range for parameters associated with hard domains.
• Benefit: It efficiently discovers the minimal sufficient randomization needed for robustness, avoiding unnecessary complexity and improving training stability compared to manually set, fixed bounds.

This is the most common application, focusing on randomizing visual appearance to force the policy to learn from geometry and semantics rather than superficial textures. Key randomized parameters include:

• Object textures and colors (e.g., applying random materials from a large library).
• Lighting conditions (intensity, direction, number of light sources, color temperature).
• Camera properties (field of view, focal length, noise, distortion).
• Background scenes (swapping out skyboxes, floor patterns, and clutter).

Example: Training a vision-based robotic grasping policy with thousands of random object colors, under starkly different lighting, ensures it learns to recognize shapes for grasping, not specific color cues that won't exist in a real warehouse.

This application randomizes the physical parameters of the simulation to create policies robust to inaccuracies in the simulated model of reality (the reality gap). Randomized parameters often include:

• Mass and inertia of objects and the robot's own links.
• Friction coefficients (surface friction, motor friction).
• Motor strength and control gains (torque limits, PID parameters).
• Actuator latency and noise.
• Object material properties (elasticity, damping).

Example: A quadruped robot trained with randomized ground friction, leg mass, and motor response will learn a locomotion policy that adapts to slippery floors, carpet, or slight hardware wear in the real world, without ever experiencing them directly.

A flagship application where simulation is essential for safety. Randomization creates a vast corner case curriculum for perception and control systems.

• Weather and Time of Day: Rain, fog, snow, blinding sun, and night conditions are simulated with varying intensity.
• Traffic Behavior: Non-player vehicle (NPC) drivers are given randomized aggressiveness, adherence to rules, and trajectory patterns.
• Road and Signage Appearance: Lane markings, traffic signs, and road surfaces are varied in color, wear, and style (e.g., European vs. North American signs).
• Sensor Noise: LiDAR point cloud density, camera grain, and GPS accuracy are perturbed.

This forces the driving policy to rely on fundamental rules (e.g., stay in lane, avoid collisions) rather than memorizing specific visual scenarios.

This is the overarching goal that unifies all applications. Domain Randomization is the primary technique for closing the sim-to-real gap. The core hypothesis is that by training across a sufficiently broad distribution of simulated domains, the real world will appear as just another (unseen) sample from that distribution.

Key Workflow:

1. Define parameter ranges for visual and dynamics properties.
2. For each training episode, sample a new randomized domain from these ranges.
3. Train the policy to maximize reward across all these domains.
4. Deploy the final policy directly on the physical system without any fine-tuning.

Successful deployment in robotics manipulation, drone flight, and legged locomotion validates this approach, though it requires careful tuning of randomization ranges.

An advanced application where Domain Randomization is coupled with online adaptation. Instead of learning a single robust policy, the system learns a policy that can quickly identify the current domain's parameters and adjust its behavior.

Process:

• Train a universal policy across a wide range of randomized dynamics.
• Simultaneously, train a context encoder network to infer the latent dynamics parameters from a short history of interactions.
• At deployment, the real robot uses the encoder to estimate the current "domain" (e.g., high friction, low battery) and conditions the universal policy accordingly.

This moves beyond pure robustness towards adaptive intelligence, allowing a single policy to perform optimally across diverse real-world conditions it was trained on in simulation.

Automatic Domain Randomization (ADR) is an algorithmic extension that addresses a key limitation: manually setting the right bounds for randomization is difficult. ADR automatically expands the range of randomization during training.

How it works:

1. Start with a narrow, easy distribution of parameters.
2. Train the policy until it succeeds in the current distribution.
3. Automatically generate new, harder environments at the boundary of the policy's capability (e.g., higher friction, weirder textures).
4. Add these to the training distribution and repeat.

This creates a curriculum of increasing difficulty, systematically probing for and strengthening the policy's weaknesses. It has been crucial for solving complex tasks like solving a Rubik's Cube with a robot hand purely in simulation before real-world transfer.

The overarching process of training a policy or model in a simulated environment and deploying it successfully on a physical system. The primary challenge is the reality gap—the discrepancy between simulation and real-world physics and visuals. Techniques like Domain Randomization, system identification, and fine-tuning are employed to bridge this gap.

The process of building an accurate mathematical model of a physical robot's dynamics (e.g., mass, friction, actuator response) to improve simulation fidelity. It is often used in conjunction with Domain Randomization:

• High-fidelity sims: Use identified parameters for a precise baseline model.
• Randomization ranges: Define plausible variation bounds around these identified parameters for robust training.

The fundamental discrepancy between a simulation's modeled environment and the real world. This gap causes policies trained purely in simulation to fail upon deployment. It has two main components:

• Visual Domain Gap: Differences in lighting, textures, and camera artifacts.
• Dynamics Domain Gap: Differences in physics, friction, mass, and actuator latency. Domain Randomization explicitly attacks this gap by training across a vast distribution of simulated worlds.

A related but distinct technique where a model is adapted from a source domain (e.g., simulation) to a specific target domain (e.g., the real world), often using unlabeled target data. Contrast with Domain Randomization:

• Domain Adaptation: Adapts to one target reality.
• Domain Randomization: Trains to be invariant across many possible realities, aiming for zero-shot transfer.

A subsequent stage where a policy pre-trained with Domain Randomization in simulation is further trained using a small amount of real-world data. This hybrid approach:

• Leverages randomization to learn robust, transferable features.
• Uses limited real data to specialize and correct for residual sim-to-real errors. It reduces the amount of costly and potentially dangerous real-world experimentation required.