Reality Gap

Simulation engines make simplifying assumptions about rigid body dynamics, contact forces, and material properties that diverge from physical reality. Key mismatches include:

• Contact and friction models that fail to capture complex surface interactions.
• Simplified actuator dynamics ignoring motor saturation, backlash, and thermal effects.
• Mass and inertia approximations that do not account for manufacturing tolerances. For example, a simulated robotic gripper may apply perfect forces, while its physical counterpart exhibits variable slip due to unmodeled surface texture.

Real-world sensors introduce stochastic noise, calibration drift, and processing latency absent in perfect simulation streams.

• Cameras suffer from motion blur, rolling shutter effects, and auto-exposure changes.
• LiDAR/Depth sensors produce spurious returns and have limited resolution.
• Inertial Measurement Units (IMUs) have bias and drift over time.
• Proprioceptive sensors (e.g., joint encoders) have quantization error. Policies trained on pristine, noiseless sensor data fail when deployed because they are not robust to this inherent signal corruption.

Simulations struggle to replicate the full complexity and non-stationarity of real-world environments.

• Lighting conditions (glare, shadows, time of day) are highly variable and affect vision systems.
• Air currents, temperature, and humidity affect aerial and lightweight robots.
• Deformable objects (cables, fabrics) and granular materials (sand, gravel) have extremely complex physics.
• Dynamic obstacles (e.g., people, other robots) behave unpredictably. This creates a distributional shift where the policy encounters states far outside its simulated training distribution.

The low-level control stack on physical hardware introduces delays, non-linearities, and bandwidth limitations not present in simulation.

• Command latency from the policy's neural network output to actual motor movement.
• PD controller tuning differences between simulated and real joints.
• Torque saturation and current limits of real motor drivers.
• Communication bus delays (e.g., CAN, Ethernet) in distributed systems. A policy issuing torque commands in simulation may assume instantaneous, perfect execution, leading to instability when faced with these real-world control loop constraints.

Many simulations provide direct access to ground truth state information (object poses, velocities) that is unavailable or noisy in reality.

• Policies may be trained on perfect pose estimates, bypassing the need for a robust perception system.
• Real perception involves segmentation, object detection, and state estimation, each adding error.
• Sim2Real for vision requires bridging the visual domain gap in textures, lighting, and object appearance. This cause is particularly acute for end-to-end policies that map pixels to actions, as they must generalize from rendered images to real camera feeds.

The computational cost of high-fidelity simulation forces pragmatic trade-offs that widen the reality gap.

• Simulation timestep is often larger than real control frequencies, missing high-frequency dynamics.
• Collision detection uses approximate geometries (convex hulls) for speed.
• Numerical integrators (e.g., Euler vs. Runge-Kutta) introduce approximation error.
• Massively parallel training often uses many low-fidelity sim instances rather than a few high-fidelity ones. These engineering decisions are necessary for feasible training times but inherently limit the simulation's accuracy.

Domain Randomization (DR) is the foundational technique for sim-to-real transfer. It trains a policy by randomizing non-essential simulation parameters—like object textures, lighting, friction coefficients, and sensor noise—across a wide range. The core hypothesis is that by exposing the model to a vast, randomized distribution of simulated worlds, it will learn a policy that is invariant to these details and thus generalizes to the unseen real world. For example, a robot arm trained with randomized lighting, object colors, and surface friction learns to grasp based on geometry and physics, not specific visual cues. This enables zero-shot transfer, where the policy works on physical hardware without any real-world fine-tuning.

This technique directly reduces the reality gap by calibrating the simulation's physics model to match data from the real robot. System Identification (SysID) involves collecting input-output data from the physical system (e.g., motor commands and resulting joint angles) and using it to fit the parameters of the simulation's dynamic model.

• Process: Execute motions on the real robot, record data, and optimize simulation parameters (mass, inertia, friction, motor gains) to minimize the difference between simulated and real trajectories.
• Benefit: Creates a higher-fidelity digital twin, narrowing the gap before applying robustness techniques like domain randomization.
• Trade-off: A perfectly calibrated sim for one robot may not generalize to other units due to manufacturing variances, hence it's often combined with bounded randomization around the identified parameters.

While domain randomization aims for robustness across many domains, domain adaptation actively tries to align the simulation (source domain) and reality (target domain). Instead of randomizing away the gap, it learns to translate or adapt features from one domain to the other.

• Feature-Level Adaptation: Uses adversarial training or discrepancy minimization to learn domain-invariant representations, so a perception module cannot distinguish whether its input is from simulation or reality.
• Pixel-Level Adaptation (Sim2Real): Employs generative models like CycleGAN to translate simulated images into photorealistic ones, or real images into simulated styles, creating a aligned dataset for training.
• Fine-Tuning: After simulation pre-training, the policy is briefly fine-tuned on a small amount of real-world data, a process also known as sim-to-real transfer learning. This is effective when limited real-world interaction is permissible.

These methods close the loop by incorporating real-world experience directly into the training process, often in a safe, sample-efficient manner.

• Residual Learning: The policy learns a residual correction on top of a simulation-trained base policy or a classical controller. Deployed on the real robot, it adapts only the necessary adjustments, reducing risky exploration.
• Iterative Training: A DAgger-style process where a simulation-trained policy is deployed, its failures are recorded, and these real-world examples are used to iteratively refine the simulation or re-train the policy.
• Meta-Learning for Fast Adaptation: The policy is meta-trained in simulation to quickly adapt to new dynamics with minimal real-world trials. When deployed, it uses a handful of real interactions to identify the current "domain" and adjust its behavior accordingly.

Advanced randomization strategies that move beyond uniform sampling to train more efficient and capable policies.

• Automatic Domain Randomization (ADR): An algorithmic method that starts with a narrow parameter range and automatically expands it only in directions where the policy is failing, creating a curriculum of complexity. This maximizes the entropy of the training distribution, systematically searching for the policy's worst-case domain.
• Curriculum Randomization: Manually designs a training schedule that starts with easy, near-nominal simulation parameters and gradually increases the difficulty and range of randomization as the policy improves.
• Bounded Randomization: Constrains randomization to physically plausible ranges (e.g., friction between 0.2 and 0.8) to prevent training on nonsensical dynamics, leading to more physically-grounded policies.

These techniques blend physical and virtual components to create a more faithful training or testing environment.

• Hardware-in-the-Loop (HITL): The real robot's controller or perception system is connected to a real-time simulation of the environment and physics. This validates low-level control and latency in a safe, repeatable setting before full deployment.
• Digital Twin with Real Data Injection: A high-fidelity digital twin is continuously updated with sensor data from the physical asset. This twin can be used to test new policies or predict failures in a perfectly synchronized virtual space.
• Hybrid Dynamics: Parts of the system are simulated (e.g., complex object interactions) while other parts use pre-recorded or simplified real-world data streams, reducing the modeling burden for the most complex phenomena.

A core technique for addressing the reality gap by training a model in a simulation where key parameters are randomly varied. This forces the model to learn a robust policy that generalizes to unseen conditions.

• Key Parameters: Includes physics (mass, friction), visuals (textures, lighting), and sensor noise.
• Goal: Achieve zero-shot transfer where the policy works on a physical robot without real-world fine-tuning.

The fundamental problem underlying the reality gap. It refers to the performance degradation when a model trained on data from a source domain (simulation) is applied to a different target domain (the real world).

• Causes: Differences in data distribution, sensor characteristics, or physical dynamics.
• Relation to Robustness: Overcoming domain shift is the goal of achieving out-of-distribution (OOD) robustness.

The process of building or calibrating a mathematical model of a physical system (like a robot) using observed data. It is used to reduce the reality gap by making the simulation more accurate.

• Contrast with DR: While domain randomization embraces inaccuracy, system identification seeks to minimize it.
• Application: Calibrating simulation parameters (e.g., motor torque constants, link masses) to match real-world hardware telemetry.

The primary quantitative metric for evaluating sim-to-real transfer. It measures the proportion of successful task executions when a simulation-trained policy is deployed on physical hardware.

• Benchmarking: A core component of sim-to-real benchmarking protocols.
• Interpretation: A high success rate indicates effective bridging of the reality gap, often through techniques like domain randomization.

The ideal outcome of sim-to-real training, where a policy trained exclusively in simulation performs successfully on a physical robot without any additional fine-tuning on real-world data.

• Achievement Method: This is the explicit goal of domain randomization and automatic domain randomization (ADR).
• Economic Impact: Enables scalable robotic training by eliminating costly and time-consuming real-world data collection.

The engineering balance between the computational cost and physical accuracy of a simulation. High-fidelity simulations are computationally expensive but may have a smaller initial reality gap.

• Domain Randomization Approach: Often uses lower-fidelity simulations that are fast but inaccurate, relying on randomization to compensate for the lack of precision.
• Practical Consideration: Drives decisions in parallelized simulation infrastructure design for training at scale.