Domain Randomization

The fundamental operation involves systematically varying parameters within a physics simulator during training. This prevents the agent from overfitting to the specific quirks of the simulation. Key randomized elements include:

• Visual properties: Object textures, colors, lighting conditions (position, intensity, color), camera noise, and background scenes.
• Physical/dynamic properties: Object masses, friction coefficients, actuator dynamics (motor strength, latency), and material elasticity.
• Geometric properties: Object shapes, sizes, and initial positions within the workspace. By training across this broad distribution of randomized environments (D_rand), the policy learns an invariant, robust strategy applicable to the real world (D_real).

Unlike striving for photorealism, domain randomization prioritizes robust feature learning. The philosophy is that it is more effective to train on many inaccurate but varied simulations than on one highly accurate simulation. The policy is forced to rely on invariant features of the task (e.g., geometric relationships, functional affordances of objects) rather than spurious visual or dynamic correlations specific to the simulator. This directly combats the reality gap—the inevitable mismatch between even the best simulation and physical reality—by making the policy resilient to unseen parameter variations.

Randomization strategies exist on a spectrum:

• Uniform Randomization: Parameters are sampled from a fixed, wide uniform distribution (e.g., lighting hue from 0-360 degrees). Simple but can be inefficient.
• Curriculum/Adaptive Randomization: The distribution of parameters is adjusted during training, often starting with a narrow, easy distribution and progressively widening it (domain randomization curriculum) or adapting it based on the agent's performance.
• Structured/Systematic Randomization: Parameters are varied in a correlated, physically plausible way rather than independently. For example, changing the sun's position should consistently affect shadow direction and intensity across all objects. This can improve sample efficiency and final performance.

These are two primary approaches to sim-to-real transfer with different mechanisms:

• Domain Randomization: Does not try to make the source (simulation) domain match the target (real) domain. Instead, it expands the source domain to be so broad that the real world appears as just another sample from the training distribution. It requires no real-world data for training.
• Domain Adaptation: Actively tries to align the source and target domains, often using techniques like adversarial training or feature alignment to minimize a measure of discrepancy (e.g., Maximum Mean Discrepancy). This typically requires some unlabeled (or sometimes labeled) real-world data during training. Domain randomization is often favored when real-world data collection is expensive or dangerous.

Domain randomization is a cornerstone technique for training visuomotor policies and perception systems for physical robots. Typical use cases include:

• Robotic Manipulation: Training grippers to pick and place objects with varied appearance, lighting, and friction.
• Autonomous Navigation: Teaching drones or ground robots to fly/drive in environments with randomized textures, lighting, and object placements.
• Perception Module Training: Generating synthetic, randomized training data for object detectors or segmenters that must work under diverse real-world conditions.
• Bridging Simulator Differences: Transferring policies trained in one simulator (e.g., MuJoCo) to another (e.g., Isaac Sim) or to real hardware.

While powerful, naive domain randomization has limitations, leading to more advanced methods:

• Inefficient Exploration: Extremely wide randomization can make learning the core task difficult. Solutions include automatic domain randomization (ADR), which actively seeks out challenging parameter ranges.
• Reality Neglect: The policy may learn to ignore all visual input if randomization is too extreme. Careful parameter selection is required.
• Dynamic Complexity: Randomizing high-dimensional physical parameters can lead to unrealistic or unstable simulations. Structured randomization addresses this.
• Combination with Other Techniques: Often used in tandem with fine-tuning on real data, distillation, or domain adaptation methods for final performance polishing.