Domain Randomization

The core mechanism involves defining a randomization distribution over every relevant simulation parameter. During each training episode, specific values are sampled from these distributions. This includes visual parameters (lighting position, color temperature, object textures, camera position, background scenes) and physical parameters (friction coefficients, joint damping, object mass, actuator latency, sensor noise). The model never sees the same exact environment twice, preventing it from overfitting to specific visual cues or dynamics. The goal is to make the real world appear to the model as just another sample from the training distribution.

The fundamental challenge in sim-to-real transfer is the reality gap—the discrepancy between simulated and physical environments caused by imperfect physics modeling, rendering artifacts, and unmodeled phenomena. Domain randomization closes this gap not by building a perfect simulator, but by training a policy that is insensitive to environmental variation. Key principles:

• The simulator does not need to be photorealistic; it needs to be diverse enough to encompass reality.
• Randomization forces the model to learn invariant features that are robust across visual and physical domains.
• The technique is particularly effective for vision-based policies where pixel-level differences between simulation and reality are most pronounced.

Naive uniform randomization can be inefficient if the model never experiences a solvable environment. Advanced strategies structure the difficulty:

• Uniform Domain Randomization (UDR): Parameters are sampled independently from fixed, wide uniform distributions throughout training.
• Curriculum Learning: Randomization ranges start narrow (easy) and progressively widen (hard) as the model improves, preventing early training collapse.
• Adaptive Domain Randomization (ADR): The randomization distribution itself is dynamically adjusted based on the model's current performance. Parameters where the model struggles are expanded; parameters where it excels are contracted. This automatically focuses computational effort on the most challenging aspects of the sim-to-real gap.

A specific subset of domain randomization focused exclusively on physical parameters rather than visual appearance. This is critical for robotic manipulation and locomotion tasks where accurate force transmission is essential. Randomized dynamics parameters include:

• Link masses and inertias: The weight distribution of robot limbs.
• Joint friction and damping: Resistance in actuators and bearings.
• Contact parameters: Stiffness and friction coefficients between gripper and object.
• Actuator dynamics: Motor torque limits, gear backlash, and control latency.
• Time step variability: Simulating inconsistent physics step sizes. A policy trained with dynamics randomization learns robust control strategies that function even when the physical robot's exact parameters are unknown or change due to wear.

Visual domain randomization systematically alters the appearance of rendered images to prevent the model from relying on simulation-specific visual artifacts. Common techniques include:

• Lighting randomization: Number of light sources, position, color, intensity, and ambient occlusion.
• Texture randomization: Replacing object textures with random patterns or colors from a large dataset.
• Camera randomization: Field of view, focal length, position, orientation, and sensor noise models.
• Background randomization: Replacing the background with random natural images or procedural noise.
• Post-processing effects: Random Gaussian blur, motion blur, color jitter, and contrast adjustments. These techniques are often sufficient to transfer a vision-based policy to reality even when trained on non-photorealistic, procedurally generated imagery.

Domain randomization is not a universal solution and has distinct failure modes:

• Unrealizable distributions: If the randomization range does not encompass the real-world parameter, the policy will fail. The real world must be a subset of the training distribution.
• Over-randomization: Excessively wide randomization can make the task impossible to learn, as no consistent signal emerges from the noise.
• Observable parameter mismatch: If the model can observe the randomized parameters directly (e.g., seeing the exact friction coefficient), it may learn to condition on this privileged information, which is unavailable in reality. Privileged information must be carefully excluded from the observation space.
• Computational cost: Training requires orders of magnitude more simulation samples than a non-randomized baseline, demanding significant GPU and simulation infrastructure.