Domain Randomization

The core mechanism involves randomizing non-essential visual and physical parameters of the simulator at the start of each training episode. This includes:

• Visuals: Lighting position, color temperature, camera angle, background textures, object colors
• Dynamics: Friction coefficients, joint damping, mass distributions, actuator latency
• Sensor Noise: Gaussian noise injection, dropout artifacts, depth sensor speckle

By seeing the same task under thousands of randomized conditions, the model learns to ignore spurious correlations and focus on the invariant causal structure of the problem.

An advanced variant where the randomization distribution itself is learned rather than manually specified. The process:

• Start with a narrow distribution of environment parameters
• Train the policy until performance plateaus
• Expand the distribution in dimensions where the policy performs well
• Contract in dimensions where performance degrades
• Repeat until the distribution covers real-world variability

This creates a curriculum learning effect, progressively exposing the agent to harder environments without requiring human intuition about parameter bounds.

Two complementary strategies are often combined:

Visual Randomization targets the perception gap:

• Randomizes textures, lighting, camera intrinsics, and backgrounds
• Forces the vision encoder to learn appearance-invariant features
• Critical when deploying with real cameras that differ from renderers

Dynamics Randomization targets the physics gap:

• Randomizes mass, friction, motor strength, joint damping, and contact parameters
• Forces the policy to be robust to modeling errors
• Essential for contact-rich tasks where simulators are least accurate

The combined approach ensures both the perception stack and control policy generalize.

From a statistical learning perspective, domain randomization acts as a powerful regularizer that prevents overfitting to simulator artifacts:

• It minimizes the maximum risk across a family of source domains, approximating robust optimization
• The randomized simulator becomes a proxy for real-world variance, forcing the model to learn a broader, flatter loss landscape
• This connects to invariant risk minimization (IRM) — the model learns representations that are stable across all training environments
• Unlike fine-tuning on real data, it requires zero real-world samples during training, making it ideal for safety-critical or expensive-to-sample domains

Despite its power, domain randomization has known failure modes:

• Unmodeled phenomena: If the simulator cannot represent certain physics (e.g., aerodynamic turbulence, fluid dynamics), randomization cannot compensate
• Over-randomization: Excessively wide distributions can make the task impossible to learn, as the signal-to-noise ratio collapses
• Visual domain gap: Photorealistic rendering remains challenging; randomized low-fidelity graphics may not cover the distribution of real camera images
• Sample inefficiency: Training across massive distributions requires significantly more simulation samples than training in a single environment

These limitations motivate hybrid approaches combining domain randomization with domain adaptation on limited real data.

The dynamic adjustment of a simulation model's complexity and resolution to balance computational cost against the required accuracy for a specific analysis. In the context of domain randomization, fidelity scaling allows engineers to train initial policies in low-fidelity, fast environments with high randomization ranges, then progressively increase fidelity for fine-tuning.

• Low Fidelity: Simplified physics, low-polygon meshes, basic lighting — used for rapid exploration
• High Fidelity: Ray-traced rendering, fluid dynamics, soft-body physics — used for final validation
• Adaptive Scaling: Automatically adjusts detail based on the agent's current learning phase

The scientific process of characterizing and reducing all sources of uncertainty in a simulation model to establish confidence bounds on its predictions. When applying domain randomization, UQ helps determine the appropriate ranges and distributions for randomized parameters, ensuring the model is exposed to realistic variability without being overwhelmed by implausible edge cases.

• Aleatoric Uncertainty: Inherent stochasticity in the environment (e.g., sensor noise) — irreducible
• Epistemic Uncertainty: Model ignorance due to lack of data — reducible with more sampling
• Application: Guides the selection of randomization distributions (uniform, Gaussian, log-normal) for each parameter

A sequential design strategy for optimizing expensive-to-evaluate black-box functions, commonly used to auto-tune simulation parameters and hyperparameters. In domain randomization workflows, Bayesian optimization can automatically discover the optimal range and schedule of randomization that maximizes transfer performance without manual trial-and-error.

• Surrogate Model: A Gaussian Process that approximates the true objective function
• Acquisition Function: Balances exploration of uncertain regions against exploitation of known high-performing regions
• Use Case: Automatically tuning the range of friction coefficients, mass values, and lighting conditions to maximize real-world policy success

A data-driven approximation of a complex, high-fidelity simulation that executes significantly faster, enabling real-time optimization and what-if analysis. Surrogate models can be trained on the outputs of domain-randomized simulations to predict transfer performance without running expensive physics engines, dramatically accelerating the sim-to-real pipeline.

• Techniques: Neural networks, Gaussian Processes, Polynomial Chaos Expansion
• Speedup: Can reduce simulation time from hours to milliseconds per evaluation
• Integration: Used as a fast proxy for evaluating policy robustness across thousands of randomized parameter combinations

A systematic method for planning simulation runs to efficiently determine the relationship between input factors and output responses with minimal computational effort. DOE provides a structured alternative to random sampling when exploring the parameter space for domain randomization, ensuring that the most informative combinations of parameters are tested.

• Factorial Designs: Test all combinations of discrete parameter levels to identify interactions
• Latin Hypercube Sampling: Efficiently covers the parameter space with fewer runs than full factorial
• Application: Identifying which randomization parameters (lighting, friction, mass) have the greatest impact on sim-to-real transfer success