Reality Gap

Simulations rely on numerical values for physics parameters like mass, inertia, friction coefficients, and motor constants. Even small errors in these values—often derived from datasheets or rough measurements—compound over time, causing trajectories and contact forces to diverge from reality. For example, a 5% error in a robot arm's link inertia can lead to significant torque prediction errors during fast movements.

To run in real-time, simulators make necessary simplifications that omit complex physical phenomena. Common unmodeled dynamics include:

• Actuator dynamics: Non-linearities in motors, gearbox backlash, and current saturation.
• Flexibility: Vibration in links, cables, and joints.
• Complex contact mechanics: Surface deformation, rolling friction, and micro-slip.
• Fluid dynamics: Air resistance and damping effects. These omissions create a simplified world where policies learn behaviors that fail under real physical complexity.

Real sensors (encoders, cameras, IMUs) introduce measurement noise, latency, and bias absent in perfect simulation outputs. Similarly, real actuators have command latency and stochastic response. A policy trained on noiseless, instantaneous state observations becomes fragile when deployed with:

• Quantization noise from digital encoders.
• Gaussian white noise in joint velocity estimates.
• Motion blur and variable exposure in cameras. This discrepancy forces the policy to handle a partially observable, noisy state estimation problem it was never trained for.

For vision-based policies, the domain gap between synthetic and real visuals is a major cause. Synthetic renderers, even photorealistic ones, struggle with:

• Lighting: Perfect global illumination vs. complex shadows, glare, and HDR.
• Textures: Repetitive, tileable materials vs. natural wear, stains, and uniqueness.
• Geometry: Perfectly meshed objects vs. manufacturing imperfections. This gap causes policies to overfit to visual artifacts of the simulator, failing to generalize to the messy aesthetics of the real world.

Simulation bias arises from the numerical integration methods and solver approximations used to compute physics. Discrete time steps, collision detection tolerances, and floating-point precision all introduce small, systematic errors. For instance:

• A fixed-time-step integrator approximates continuous dynamics, accumulating energy drift.
• Penalty-based contact solvers can produce overly "sticky" or "bouncy" interactions compared to real impacts. These biases are baked into every training episode, teaching the policy the "laws" of the simulator, not the laws of physics.

Simulations are often deterministic or have limited variability, while the real world is inherently stochastic. A policy trained in a clean, controlled virtual lab faces unpredictable real-world environmental stochasticity:

• Variable floor friction from dust or moisture.
• Changing payloads and object properties.
• Unmodeled external disturbances like vibrations or air currents.
• Human presence and interaction. The policy's lack of exposure to this breadth of conditions during training is a primary source of the transfer error upon deployment.

A domain gap is the statistical difference between the data distribution of a source environment (e.g., simulation) and a target environment (e.g., reality). This is the fundamental cause of the reality gap, manifesting as differences in:

• Visual appearance: Lighting, textures, and rendering artifacts.
• Dynamics: Friction, actuator response, and contact modeling.
• Sensor noise: Imperfections in simulated vs. real camera, LiDAR, or IMU data. Techniques like domain adaptation and domain randomization are explicitly designed to bridge this gap.

Simulation bias is a systematic error introduced by the approximations, assumptions, and numerical methods inherent in a simulator. Unlike random noise, it causes the simulator's predictions to consistently deviate from real-world behavior. Key sources include:

• Simplified physics: Using rigid-body dynamics instead of modeling soft-body deformations.
• Numerical integrators: Discretization errors from solvers like Runge-Kutta.
• Contact model approximations: Imperfect collision detection and resolution. This bias directly widens the reality gap and must be characterized through quantitative validation.

Unmodeled dynamics are physical phenomena or system behaviors not captured by the mathematical model used for simulation or control. These are a primary contributor to the reality gap, as the policy never encounters these effects during training. Common examples in robotics are:

• Cable dynamics: Forces from dangling wires or hoses.
• Actuator saturation and hysteresis: Non-linearities in motors and gears.
• Structural flexibility: Vibration and bending in "rigid" links.
• Fluid dynamics: Air resistance or hydraulic effects. Mitigation involves residual modeling or designing robust controllers.

Model uncertainty quantifies the lack of perfect knowledge about a system's true dynamics. It encompasses both epistemic uncertainty (from limited data or simplifications) and aleatoric uncertainty (inherent stochasticity). In the context of the reality gap:

• It arises from inaccurate physics parameters, unmodeled dynamics, and sensor noise models.
• It forces policies to be robust to a distribution of possible dynamics, not a single model. Techniques like Bayesian calibration and domain randomization explicitly account for this uncertainty to improve sim-to-real transfer.

Transfer error is the quantitative performance loss observed when a policy trained in simulation is deployed on a real physical system. It is the measurable consequence of the reality gap. It can be expressed as:

• Task-specific metrics: Decreased success rate, increased time-to-completion.
• Control metrics: Higher tracking error, increased control effort (torque/energy).
• Stability metrics: More frequent failures or crashes. Reducing transfer error is the ultimate goal of sim-to-real transfer methods like system identification and adaptive control.

System identification is the process of constructing mathematical models of dynamic systems from measured input-output data. It is a primary engineering method for reducing the reality gap by making the simulation more accurate. The pipeline involves:

• Designing excitation trajectories to persistently excite all system modes.
• Collecting real-world data (joint positions, torques, accelerations).
• Parameter estimation to calibrate simulation constants (mass, friction).
• Validation against a separate dataset. Grey-box identification combines this with data-driven residual modeling for highest fidelity.