Domain Randomization

The reality gap is the fundamental discrepancy between a simulated training environment and the real world. This gap arises from imperfect modeling of physics, sensor noise, material properties, and environmental conditions. Domain Randomization directly attacks this problem by ensuring the agent never overfits to a single, inaccurate simulation. By training across a distribution of simulations, the agent learns policies that are robust to the uncertainties inherent in any single model of reality.

This is the core mechanism: systematically varying parameters of the simulation during every training episode. Key randomized domains include:

• Visual Dynamics: Object textures, colors, lighting conditions, and camera noise.
• Physical Dynamics: Mass, friction, damping coefficients, and actuator delays.
• Environmental Geometry: Object shapes, sizes, and initial positions.
• Task Configuration: Goal locations and disturbance forces.

The agent must succeed despite these variations, learning a policy that relies on invariant features rather than simulation-specific artifacts.

Two primary strategies define the sampling of parameters:

• Uniform Randomization: Parameters are sampled from a fixed, wide uniform distribution (e.g., friction between 0.5 and 1.5). Simple and effective, it assumes covering a broad range will encompass reality.
• Adaptive Randomization (Domain Adaptation): The distribution of parameters is dynamically adjusted based on agent performance, often using a teacher-student setup or Bayesian optimization. The goal is to progressively narrow the randomization to more realistic regions or to focus on challenging edge cases, improving sample efficiency.

This distinction concerns the structure of the variation:

• Systematic Randomization: Parameters are varied along structured axes in a grid-like fashion (e.g., low/medium/high friction combined with low/medium/high mass). This ensures coverage but can be combinatorially expensive.
• Random Randomization: Parameters are sampled independently and randomly from their distributions for each training episode. This is more computationally efficient and can produce a vast, continuous set of environments, but may leave gaps in the coverage of the parameter space.

Domain Randomization transforms the reinforcement learning objective. Instead of maximizing reward in a single deterministic environment M, the agent must maximize expected reward across a family of environments M_θ, where θ represents the randomized parameters. The policy π is optimized for: J(π) = E_θ~P(Θ)[ E_τ~π, M_θ [ Σ γ^t r_t ] ] where P(Θ) is the randomization distribution. This forces the policy to be robust to the entire distribution, not just its mean.

While powerful, naive Domain Randomization has limits. Excessively wide randomization can make learning impossible. Advanced techniques address this:

• Curriculum Randomization: Start with narrow, easy distributions and gradually widen them as the agent learns.
• Simulation-to-Simulation Transfer: Train in one physics engine (e.g., PyBullet) with randomization and test in another (e.g., MuJoCo) to verify robustness isn't engine-specific.
• Combination with System Identification: Use brief real-world data to calibrate the center of the randomization distribution, reducing the required range. These methods bridge the gap between pure simulation and targeted real-world adaptation.

Generates diverse synthetic medical data where real data is scarce, privacy-sensitive, or difficult to label.

• Surgical Simulation: Training robotic surgery policies by randomizing tissue properties (elasticity, deformation), blood vessel textures, and tool-tissue interaction dynamics in a virtual operating room.
• Medical Image Synthesis: Creating synthetic MRI, CT, or X-ray images with randomized anatomical variations, pathologies, and imaging artifacts (noise, motion blur) to augment datasets for diagnostic model training while preserving patient privacy.
• Domain Adaptation: Using randomized synthetic data to adapt models trained on data from one hospital scanner to work effectively on data from another scanner with different imaging characteristics.

Enables the training of robust visual inspection systems in manufacturing without requiring thousands of physical defect samples.

• Defect Synthesis: Randomizing the appearance, size, location, and type of defects (scratches, dents, discolorations) on 3D models of manufactured parts.
• Environmental Variation: Simulating changes in factory lighting, camera angles, part orientation, and surface wear to ensure the inspection model works reliably on the production line.
• This drastically reduces the time and cost associated with collecting and labeling rare defect examples in the real world.

The overarching goal of training a model in simulation and deploying it in the physical world. Domain randomization is a primary technique to achieve this by exposing the agent to a vast distribution of simulated conditions, forcing it to learn invariant features that hold true across the reality gap. Success is measured by the policy's performance degradation when moving from sim to real.

The software core of a simulated environment that calculates object motion and interaction based on physical laws. For domain randomization, key parameters within the engine are varied, including:

• Mass and inertia of objects
• Coefficients of friction and restitution
• Gravity and other force magnitudes
• Actuator dynamics (e.g., motor strength, latency) Engines like NVIDIA Isaac Sim, PyBullet, and MuJoCo provide APIs for this parameter randomization.

The generation of synthetic readings from virtual sensors (e.g., RGB-D cameras, LiDAR, IMUs). Domain randomization is heavily applied here to bridge the perception gap. Common randomized attributes include:

• Camera parameters: focal length, distortion, noise models
• Lighting conditions: intensity, direction, color temperature
• Textures and materials of objects and backgrounds
• Post-processing effects: motion blur, bloom, grain This teaches the agent's perception stack to be invariant to specific visual artifacts.

An algorithmic method for creating data with near-infinite variation. While domain randomization varies parameters of a fixed environment, procedural generation often creates entirely new structures (e.g., maze layouts, building interiors, terrain meshes). The two techniques are frequently combined: a procedurally generated level then has its physical and visual parameters randomized, creating an exceptionally broad and challenging training distribution.

The inverse problem of domain randomization. Instead of randomizing simulation parameters to cover many possible realities, system identification aims to precisely calibrate a simulation's parameters to match a specific real-world system. This is used for creating high-fidelity digital twins. The approaches represent two ends of a spectrum: randomization for robustness, identification for precision.

The fundamental discrepancy between simulation and reality that causes policy failure. It has two main components:

• Dynamics Gap: Differences in physical interaction (e.g., friction, contact forces).
• Perception Gap: Differences in sensor observations (e.g., lighting, texture). Domain randomization directly attacks this gap by treating the real world as just another random sample from the broad distribution seen in training. The technique does not seek to perfectly model reality, but to make the policy indifferent to the specifics of any single model.