Domain Randomization

The systematic variation of visual properties in the simulation to force the policy to learn from essential geometric features rather than superficial textures. This is a primary method for bridging the reality gap in visual perception.

• Textures & Colors: Randomizing object and background surface patterns and hues.
• Lighting Conditions: Varying the number, position, color, and intensity of light sources.
• Camera Properties: Introducing noise, distortion, and varying focal lengths to mimic different real-world sensors.

By training across this vast visual parameter space, the model becomes invariant to the specific rendering style of the simulator, focusing instead on shape, depth, and object relationships.

The variation of physical parameters governing motion and interaction to create a robust controller that adapts to real-world uncertainty. This addresses inaccuracies in the simulator's rigid-body dynamics model.

• Mass & Inertia: Randomizing the mass distribution of objects and robot links.
• Friction Coefficients: Varying surface friction (static and dynamic) for objects and the robot.
• Actuator Models: Perturbing motor gains, latency, and torque limits to account for imperfect real-world actuators.
• Gravity & External Forces: Applying small, randomized forces to simulate air currents or uneven surfaces.

This teaches the policy to be robust to the sim-to-real transfer challenge where simulated dynamics never perfectly match physical hardware.

These are two complementary approaches for sim-to-real transfer. Domain randomization assumes the real world is an instance within a broad, randomized parameter distribution. In contrast, system identification aims to precisely calibrate the simulation to match a specific real system.

• Domain Randomization: Trains a single, robust policy across a wide distribution. It's often more practical when precise system identification is difficult.
• System Identification: Uses real-world data to tune simulator parameters, then trains a policy in this tuned, high-fidelity simulation. It can yield higher performance if identification is accurate.

Advanced approaches combine both: using a narrow, identified parameter distribution as the center of a broader randomization range.

Advanced techniques that structure the randomization process to improve learning efficiency and final performance.

• Curriculum Randomization: Starts training in a simple, narrow parameter distribution (e.g., consistent lighting) and gradually expands the randomization range as the policy improves. This prevents the policy from failing initially in an overwhelmingly complex space.
• Adaptive Domain Randomization (ADR): Automatically identifies and focuses randomization on parameters where the reality gap is largest. The system detects scenarios where the simulation-trained policy fails on real data, then increases randomization on the corresponding parameters in simulation.

These methods move beyond naive uniform randomization to a more intelligent, data-driven allocation of simulation diversity.

The deliberate injection of noise and distortion into simulated sensor readings to mimic the imperfections of real-world hardware. This is critical for policies that rely on proprioception or exteroceptive sensors.

• Proprioceptive Noise: Adding bias and Gaussian noise to joint position, velocity, and torque readings.
• LiDAR & Depth Sensor Noise: Simulating ray dropouts, spurious returns, and distance measurement errors.
• IMU Noise: Injecting drift and bias into simulated accelerometer and gyroscope readings.
• Communication Latency: Randomizing delays between sensor measurement and policy input to account for real-time system jitter.

This ensures the policy learns to filter noise and does not become reliant on the artificially clean data provided by a perfect simulator.

The reality gap is the fundamental discrepancy between simulation and reality. Domain randomization works not by eliminating this gap, but by making the policy's performance insensitive to it.

Mechanism: By training across a vast, randomized distribution of possible worlds (P_rand), the policy is forced to find a solution that works for all of them. The core assumption is that the real world (P_real) is an unknown but likely member of this broad distribution: P_real ∈ P_rand.

Key Insight: The policy learns to rely on invariant features (e.g., object geometry, task dynamics) that are consistent across the randomization, while ignoring highly variable features (e.g., exact RGB pixel values, precise friction coefficient). This emergent robustness is the key to successful sim2real transfer.

Randomizes the visual properties of objects and scenes to prevent the policy from overfitting to specific textures, colors, or lighting conditions.

• Textures & Materials: Object surface properties (e.g., wood, metal, plastic) and patterns are sampled from large, diverse libraries.
• Colors: Hue, saturation, and brightness of objects, floors, and walls are varied.
• Lighting: Number, type (directional, point, spot), intensity, color temperature, and position of light sources are randomized. Shadows and global illumination are also varied.
• Backgrounds: The scene backdrop or environment skybox is changed to include indoor settings, outdoor scenes, or abstract patterns.

Example: Training a vision-based grasping policy with objects that appear as red plastic, blue metal, or green matte in scenes lit by either a single bright overhead light or multiple dim, colored lights.

Randomizes the physical laws and object properties within the simulator to encourage policies that are robust to inaccuracies in the simulated physics model.

• Mass & Inertia: The mass and inertial distribution of manipulated objects and robot links are varied.
• Friction Coefficients: Static and dynamic friction between contacting surfaces (e.g., gripper-object, object-table) are randomized.
• Motor Models: Actuator parameters like maximum torque, velocity limits, and PID gain constants are perturbed.
• Gravity: The magnitude and sometimes direction of the gravitational vector can be slightly altered.
• Object Dimensions: The size and shape of task-relevant objects are changed within a plausible range.

Example: Training a door-opening policy where the door's hinge friction, mass, and handle size change in every episode, and the robot's own joint motors have slightly different torque outputs.

Injects realistic noise and artifacts into simulated sensor readings to mimic the imperfections of physical sensors.

• Camera Noise: Adds Gaussian noise, salt-and-pepper noise, motion blur, or defocus blur to RGB images.
• Depth Sensor Noise: Introduces quantization noise, smoothing, and missing pixels (dropout) to depth maps, simulating real LiDAR or depth cameras.
• Proprioceptive Noise: Adds bias and Gaussian noise to joint position and velocity readings.
• Latency: Simulates small, variable delays in sensor feedback to mimic real-world processing pipelines.

Example: A policy relying on a depth camera receives images where depth values are quantized, have random dropout regions, and are overlaid with mild Gaussian noise, varying in intensity each episode.

Randomizes the geometric layout and composition of the training environment to prevent spatial overfitting.

• Object Placement: The initial position and orientation (pose) of all objects in the scene are randomized within a defined workspace.
• Obstacle Introduction: The number, size, shape, and location of distractor objects (not directly part of the task) are varied.
• Camera Pose: The position and orientation of the robot's egocentric camera(s) are perturbed.

Example: For a tabletop stacking task, the blocks are scattered in different locations on each reset, the table size changes, and distractor objects like cups or cones are randomly placed around the workspace.

Randomizes parameters unique to the target application domain to handle edge cases and environmental variability.

• Clothing Dynamics (for manipulation): Fabric material properties like stiffness and damping are varied.
• Fluid Properties (for pouring): Viscosity and density of simulated liquids are changed.
• Terrain Properties (for locomotion): Friction, roughness, and incline of the ground plane are randomized.
• Deformable Objects (for manipulation): Parameters controlling an object's stiffness and plasticity are varied.

Example: Training a legged robot to walk across terrain where ground friction, slope, and small obstacle heights are randomly generated for each training run.

Describes the methodological approaches for how parameters are varied during the training process.

• Uniform Randomization: Parameters are sampled from a uniform distribution between set bounds (e.g., friction between 0.2 and 1.0). This is the most common baseline.
• Curriculum Randomization: The range or complexity of randomization is gradually increased over time, starting with a narrow, easy distribution and expanding to a broad, challenging one.
• Simulation-to-Reality Adaptation: Randomization ranges are initially broad but are later fine-tuned or adapted based on limited real-world data to better match the target domain's specific distribution.
• Systematic vs. Random: Parameters can be changed every episode, every timestep, or held constant for a fixed number of episodes to study the effect on generalization.

Core Principle: The key is to create a distribution of simulated environments so broad that the real world appears as just another sample from it.

The reality gap is the performance discrepancy between a robotic policy trained in simulation and its execution on physical hardware. It arises from inevitable modeling inaccuracies in the simulator, such as:

• Simplified contact and friction dynamics
• Imperfect actuator and sensor noise models
• Unmodeled material properties and environmental disturbances Domain randomization is a primary technique designed to bridge this gap by exposing the policy to a vast distribution of simulated conditions, forcing it to learn robust, invariant features.

Sim2Real refers to the overarching engineering challenge and suite of techniques for deploying models or policies trained in simulation onto physical robots. It is the parent problem that domain randomization aims to solve. Key methodologies include:

• System identification: Precisely tuning simulator parameters to match real-world dynamics.
• Domain adaptation: Using real-world data to adapt a simulation-trained model.
• Domain randomization: Deliberately varying parameters during training to encourage generalization. The goal is to leverage cheap, parallelizable simulation for training while achieving reliable real-world performance.

A physics engine is the core software library that performs the numerical calculations for a robotic simulation. It models Newtonian mechanics, including:

• Rigid-body dynamics: Motion of non-deformable objects under forces/torques.
• Collision detection: Identifying geometric intersections between objects.
• Contact dynamics: Resolving forces from collisions and persistent contact. Engines like MuJoCo, Bullet (via PyBullet), and NVIDIA PhysX (in Isaac Sim) provide the foundational environment where domain randomization is applied to parameters like mass, friction, and motor gains.

System identification is the process of estimating the dynamic parameters of a real-world robot or environment to improve the accuracy of its simulated model. It is often used in conjunction with or as an alternative to domain randomization.

• Process: Collect data from physical hardware (e.g., joint positions, torques) and optimize simulator parameters to minimize the difference between simulated and real trajectories.
• Contrast with Domain Randomization: System identification seeks a single, accurate model, while domain randomization trains across a broad distribution of models to achieve robustness, often tolerating less individual accuracy.

Hardware-in-the-Loop (HIL) simulation is a validation methodology where physical robotic hardware (e.g., a real embedded controller, sensor, or actuator) is connected to and interacts with a real-time simulated environment. It serves a different purpose than domain randomization:

• Primary Use: Final-stage testing and verification of control software with real hardware interfaces, before full physical deployment.
• Relation to Domain Randomization: HIL testing can be used to validate policies trained using domain randomization, ensuring they send correct commands to the actual hardware controller under realistic timing constraints.

A digital twin is a high-fidelity, continuously updated virtual replica of a physical asset, system, or process. It contrasts with a training simulation used for domain randomization:

• Digital Twin Goal: Mirror a specific real instance for monitoring, prediction, and control. It is fed live sensor data.
• Domain Randomization Goal: Create a distribution of varied, non-specific instances for training robust policies. While a high-fidelity digital twin could be used as a basis for simulation, domain randomization intentionally deviates from a single accurate model to promote generalization.