Domain Randomization

This principle involves randomizing visual properties of the simulation to prevent the policy from overfitting to specific textures, colors, or lighting conditions. The goal is to learn visual features that are invariant to appearance.

• Examples: Randomizing object textures, background scenes, lighting direction and intensity, camera noise, and post-processing effects.
• Purpose: Forces the policy to rely on geometric shapes and spatial relationships rather than superficial visual cues, making it robust to the reality gap in visual perception.

This principle randomizes the physical parameters governing the simulation's engine to account for inaccuracies in modeling real-world physics. The policy learns to adapt to a distribution of dynamics.

• Examples: Varying object masses, friction coefficients, motor torque limits, actuator delays, and gravity.
• Purpose: Encourages the learning of robust control strategies that are stable across a wide range of physical conditions, compensating for the sim2real gap in system dynamics.

Effective domain randomization requires a principled strategy for sampling simulation parameters from predefined distributions, rather than using a single fixed simulation.

• Uniform Sampling: Parameters are drawn from uniform distributions within plausible bounds (e.g., lighting intensity between 50-150 lux).
• Curriculum Strategies: Parameter ranges can be gradually expanded during training, starting with easier, narrower distributions.
• Key Insight: The diversity of the training distribution is more critical than the fidelity of any single simulation instance.

The core learning objective induced by randomization is for the neural network to discover and utilize invariant representations—features that are consistent across all randomized variations and correlate with successful task completion.

• Mechanism: By making nuisance variables (like color) unpredictable, the policy's feature extractor is pressured to discard them and focus on task-relevant features (like edges and depth).
• Outcome: This leads to policies that exhibit strong generalization to unseen real-world conditions, as they have learned a more fundamental model of the task.

Domain Randomization reframes the sim2real gap not as a single, unbridgeable discrepancy, but as a distribution of possible realities. Training across a broad distribution aims to make the real world just another sample.

• Philosophy: Instead of perfecting one simulation, create many imperfect simulations whose collective diversity envelops the target real environment.
• Limitation: If the real world contains phenomena outside the sampled distribution (e.g., a completely novel object type), generalization may fail, highlighting the need for careful distribution design.

Often used in contrast or conjunction with Domain Randomization, System Identification involves precisely calibrating a single simulation to match the dynamics of a specific real robot.

• Comparison: Domain Randomization embraces uncertainty and trains for robustness across many possibilities. System Identification reduces uncertainty by carefully measuring and modeling one reality.
• Hybrid Approach: A common strategy is to use system identification to establish a nominal parameter set, then apply randomization around those values to account for residual uncertainty and wear over time.

The fundamental principle is to randomize non-essential simulation parameters across every training episode. This creates a broad distribution of visual and dynamic experiences. The policy is incentivized to ignore these varying, randomized features and focus on the invariant, task-relevant cues that are consistent with the real world.

• Visual Randomization: Textures, colors, lighting (position, intensity, color), camera noise, and background scenes.
• Dynamic Randomization: Physics parameters like object mass, friction coefficients, motor torque limits, and actuator latency.
• Domain Shift: The gap between simulation (source domain) and reality (target domain) is treated as just another variation within the training distribution.

This focuses on varying the visual input to the policy's perception system, decoupling policy learning from specific textures or lighting conditions.

• Object & Texture Swapping: Applying random HDRi environment maps, swapping object surface materials (e.g., wood, metal, plastic), and using random colors.
• Lighting Perturbation: Randomizing the number, position, color temperature, and intensity of light sources in the scene.
• Camera Effects: Adding synthetic noise, blur, saturation shifts, and contrast variations to simulated camera images.
• Background Clutter: Placing random distractor objects in the scene that are irrelevant to the task.

Example: A policy trained to grasp a cube sees it rendered as red plastic under warm light in one episode, and as blue metal under cool light with shadows in the next.

This randomizes the underlying physics model of the simulation, making the policy robust to inaccuracies in the simulated dynamics and variations in the real system's physical properties.

• Mass and Inertia: Randomizing the mass and inertial properties of objects and robot links.
• Friction Coefficients: Varying surface friction (static and dynamic) for objects and grippers.
• Actuator Models: Injecting noise into motor commands, varying PID gains, and simulating latency or backlash in joint movements.
• Object Dimensions: Slightly randomizing the size and shape of task-relevant objects.

Result: The policy learns control strategies that are stable across a family of possible dynamics, rather than overfitting to one precise simulation model.

An advanced technique where a distribution over plausible real-world system parameters is estimated or defined. Training occurs across this entire distribution.

1. Parameter Bounding: Define plausible min/max ranges for key real-world parameters (e.g., gear ratio error ±5%, link length tolerance).
2. Uniform Sampling: Each training episode samples physics parameters uniformly from these bounds.
3. Policy Robustness: The final policy must perform adequately for any parameter set within the bounds, ensuring it works for the single, unknown true parameter set of the real robot.

This method explicitly bridges the sim2real gap by treating the real world as an unknown point within the trained distribution.

It's critical to distinguish Domain Randomization from Domain Adaptation.

• Domain Randomization (DR): Broadens the source domain. Assumes the real world lies somewhere within the vast, randomized training distribution. No real data is used during training.
• Domain Adaptation (DA): Narrows the gap. Uses samples from the real target domain (e.g., real images) to adapt a model trained on a specific source domain (simulation). It actively tries to align the feature spaces.

DR Pros: Simpler, requires no real-world data collection for training. DR Cons: Can be less sample-efficient; if randomization is too extreme, the policy may learn overly conservative or simplistic behaviors.

The fundamental performance discrepancy between a policy trained in a simulated environment and its behavior when deployed on a physical robot in the real world. This gap arises from modeling inaccuracies in the simulator's physics, rendering, and sensor models. Domain Randomization is a primary strategy to bridge this gap by forcing the policy to learn features invariant to these simulation imperfections.

The process of building or calibrating a simulation's dynamic model by estimating its physical parameters (e.g., friction coefficients, motor gains, link masses) from real-world data. It is often contrasted with Domain Randomization:

• System ID seeks a single, accurate simulation model.
• Domain Randomization embraces uncertainty by training across a distribution of plausible models, often being more robust to residual miscalibration.

A machine learning paradigm where an agent learns to make decisions by interacting with an environment to maximize cumulative reward. Domain Randomization is frequently applied within RL training loops in simulation. By randomizing dynamics and visuals during this training, the resulting RL policy becomes robust, enabling successful sim-to-real transfer for tasks like robotic manipulation and locomotion.

A branch of reinforcement learning where the agent learns or is given an internal model of the environment's dynamics. Domain Randomization can be applied to the learned dynamics model itself, creating an ensemble of plausible forward models. Planning or training with this randomized ensemble encourages policies that are robust to real-world dynamics uncertainty, a technique sometimes called Model-Based Domain Randomization.

A training strategy where tasks are presented to a learning agent in order of increasing difficulty. It can be combined with Domain Randomization via a Progressive Network or scheduled randomization:

• Start training in a simple, deterministic simulation.
• Gradually increase the range of randomization (e.g., widen physics parameter bounds, add visual distractors).
• This structured approach can lead to more stable learning and better final performance than applying full randomization from the start.

The ability of a trained machine learning model to perform accurately on new, unseen data drawn from the same underlying distribution as the training data. In robotics, Domain Generalization is the goal—performing well in any real-world environment. Domain Randomization explicitly tackles this by treating the simulation as a source of unlimited, varied training data, pushing the policy to learn the invariant core of the task.