Domain Gap

The core statistical phenomenon where the joint probability distribution of inputs and outputs in the source domain (simulation) differs from the target domain (reality). This is often decomposed into:

• Covariate Shift: Change in the distribution of input features (e.g., different lighting, textures).
• Label Shift: Change in the distribution of output labels conditional on the input.
• Concept Shift: Change in the relationship between inputs and outputs (e.g., the same visual object may have different physical properties). This shift causes models trained on source data to perform poorly on target data, as their underlying statistical assumptions are violated.

A systematic error introduced by the approximations and assumptions inherent in the simulator's physics engine, rendering engine, and asset models. Key sources include:

• Physics Simplifications: Use of rigid-body dynamics ignoring material flex, simplified contact models, or perfect actuators.
• Visual Approximations: Use of synthetic textures, perfect lighting models, and absence of sensor noise like motion blur or rolling shutter.
• Temporal Discretization: Numerical integration errors from fixed time-steps that do not capture high-frequency dynamics. Unlike random noise, this bias causes consistent, directional errors between simulated predictions and real-world observations.

Physical phenomena or system behaviors that are entirely absent from the simulation's mathematical model. These are gaps in the model's representational capacity, not just inaccurate parameters. Common examples in robotics are:

• Actuator Dynamics: Backlash, stiction, and thermal effects in motors and gearboxes.
• Environmental Interactions: Complex fluid dynamics (air resistance), deformable terrain, or cable dynamics.
• Sensor Latency and Noise: True characteristics of LiDAR, IMU, or camera sensors that are often idealized. Policies trained in simulation can develop a reliance on the simplified dynamics, leading to catastrophic failures when encountering these unmodeled effects.

A discrepancy in the underlying causal structure or mechanisms between the simulated and real environments. The simulator may model spurious correlations that do not hold in reality. For example:

• A policy may learn to associate a specific visual texture of a floor with a high-friction coefficient, while in reality, texture and friction are independent.
• In simulation, object weight and color might be artificially correlated due to asset library design, but this correlation is non-existent physically. This misalignment makes models brittle, as they learn shortcuts based on non-causal features that fail upon transfer.

The difference in the amount and quality of state information available to the agent in simulation versus reality. Simulations often provide perfect, noiseless, and complete state information (privileged information), while real sensors provide only a partial, noisy view. This gap manifests in two ways:

• State Estimation Gap: The agent must use a perception system (e.g., SLAM, object detection) in reality to estimate what is given directly in sim.
• Information Asymmetry: Latent variables (e.g., battery level, internal temperature) may be fully observable in simulation but hidden or indirectly inferred in reality. This forces a policy to rely on a perception module it was never trained with.

The gap arising from the failure of a simulation to correctly model the emergent behaviors that result from the composition of many simple, accurately modeled parts. Individual components (joints, sensors) may be simulated with high fidelity, but their complex interaction in a closed-loop system produces qualitatively different aggregate behavior. Examples include:

• Resonant Frequencies: The complex vibrational modes of a full robot assembly.
• Thermal Coupling: Heat from one motor affecting a nearby sensor.
• Control-Structure Interaction: Vibrations induced by controller actions feeding back into the system. This makes system-level validation essential, as component-level accuracy is insufficient.

The most apparent domain gap stems from differences in visual rendering between simulation and reality. This includes:

• Texture and Material Fidelity: Simulated surfaces often lack the microscopic imperfections, wear, and complex reflectance properties (e.g., subsurface scattering) of real materials.
• Lighting and Shadows: Global illumination, realistic shadows, and complex light interactions (like caustics) are computationally expensive to simulate in real-time, leading to simplified lighting models.
• Sensor Noise and Artifacts: Real cameras exhibit noise, motion blur, lens distortion, and auto-exposure effects that are often absent or overly simplified in synthetic images.
• Object and Background Diversity: Simulated training environments typically have fewer object models and less cluttered backgrounds than the infinite variability of the real world.

This gap arises from inaccuracies in modeling the physical laws governing motion and interaction. Key discrepancies include:

• Contact and Friction Modeling: Simulating the complex, non-linear forces during contact, collision, and sliding (with stiction and viscous friction) is notoriously difficult. Real-world friction coefficients are often poorly known and variable.
• Actuator Dynamics: Real motors have bandwidth limits, backlash, saturation, and non-linear torque-speed curves that are frequently idealized in simulation.
• Deformable Objects and Fluids: Simulating soft bodies, cables, or fluids with real-time physical accuracy remains a major challenge, often forcing the use of rigid approximations.
• Unmodeled Effects: Aerodynamic drag, vibration, thermal expansion, and cable management forces are often omitted from simulation models, constituting unmodeled dynamics.

This gap occurs when virtual sensors do not accurately replicate the data characteristics of their physical counterparts. Examples include:

• Proprioceptive Sensors: Simulated joint encoders and torque sensors are often noiseless and perfectly calibrated, unlike real sensors which have quantization error, bias, and drift.
• Depth and LiDAR Sensors: Simulating realistic LiDAR point clouds requires accurate modeling of beam divergence, multiple-return signals, and material-dependent reflectivity, which is often approximated.
• Tactile and Force-Torque Sensing: Reproducing the high-frequency, multi-axis noise and cross-talk of a real 6-DoF force-torque sensor or tactile array in simulation is highly complex.
• Temporal Synchronization: In simulation, sensor data is perfectly synchronized, whereas real systems have variable and often unknown latencies between different sensor streams.

A critical yet often overlooked gap exists in how systems are initialized and how failed episodes are reset. In simulation:

• Deterministic Resets: A robot or object can be teleported to an exact starting position and velocity with perfect precision for every training episode.
• Known Ground Truth: The simulator has perfect knowledge of all object poses, which is used for reward calculation and supervision.
• Impossible Configurations: Simulated resets can place objects in physically unstable or inter-penetrating states that would be impossible in reality. This creates a simulation bias where the policy learns to rely on perfect knowledge and deterministic starts, which do not exist during real-world deployment, leading to poor robustness.

The definition of the task itself can differ between simulation and reality, creating a specification gap.

• Reward Function Engineering: The reward signal used to train a policy in simulation is a mathematical proxy for task success. Imperfections or "reward hacking" opportunities in this function can lead to policies that perform well in simulation but fail on the real task.
• Goal Representation: How a task goal is specified (e.g., a target position in meters) may not directly translate to real-world sensing modalities.
• Environmental Disturbances: Real-world environments have unpredictable disturbances—air currents, floor vibrations, human interaction—that are difficult to comprehensively model in simulation.
• Wear and Tear: A policy trained in simulation assumes a pristine, factory-new robot. Real systems experience mechanical wear, loosening joints, and calibration drift over time, changing the underlying dynamics.

A meta-example of a domain gap occurs not just between sim and real, but between different simulation engines. A policy trained in one physics simulator (e.g., MuJoCo) often fails when evaluated in another (e.g., PyBullet or NVIDIA Isaac Sim) due to:

• Different Numerical Integrators: Solvers use distinct algorithms (e.g., symplectic Euler vs. Runge-Kutta) with varying stability and accuracy.
• Default Parameter Values: Each simulator has different default values for global parameters like gravity, solver iterations, and contact parameters.
• Modeling Assumptions: Fundamental choices in modeling contact (e.g., spring-damper vs. complementarity constraints) create different dynamic behaviors. This highlights that the domain gap is a general problem of distributional shift, not unique to the sim-to-real transition. Techniques like domain randomization are often first validated in a Sim2Sim setting before real-world testing.

The reality gap is the performance discrepancy observed when a policy or model trained in simulation is deployed on the corresponding real-world physical system. It is the practical manifestation of the domain gap.

• Key Difference: While the 'domain gap' describes the statistical difference in data distributions, the 'reality gap' quantifies the resulting performance loss (e.g., task failure rate, control error).
• Measurement: Typically measured by the drop in key performance indicators (KPIs) like success rate, reward, or accuracy upon physical deployment.

Simulation bias is a systematic error introduced by the approximations, assumptions, and numerical methods inherent in a simulator. It causes the simulator's predictions to consistently deviate from real-world behavior, directly contributing to the domain gap.

• Sources: Includes simplified contact models, perfect sensor noise models, idealized actuator dynamics, and coarse-grain physics approximations.
• Impact: Creates a distributional shift where the simulator's state-action-next-state transitions differ from those in reality, misleading the learning algorithm.

Unmodeled dynamics are physical phenomena or system behaviors not captured by the mathematical model used for simulation or control. These missing dynamics are a primary source of the domain gap.

• Examples: High-frequency structural vibrations, complex fluid-structure interactions, non-linear friction stiction, actuator backlash, and thermal effects.
• Consequence: The simulator cannot generate data from these missing dynamics, so the trained policy is unprepared for them, leading to poor transfer and potential instability.

Model uncertainty quantifies the lack of perfect knowledge about a system's true dynamics. It encompasses both epistemic uncertainty (from limited data or knowledge) and aleatoric uncertainty (inherent randomness).

• Relation to Domain Gap: The domain gap is, in part, a manifestation of epistemic model uncertainty—the simulator is an uncertain approximation of reality.
• Mitigation: Techniques like Bayesian neural networks or ensemble methods can help policies become robust to this uncertainty, narrowing the effective gap.

Parameter calibration is the process of adjusting the numerical values of a simulation model's physics parameters (e.g., friction coefficients, masses, motor constants) to minimize the discrepancy between its predictions and observed real-world data. It is a direct method for reducing the domain gap.

• Process: Uses system identification techniques on data collected from the real system.
• Limitation: Can only adjust parameters within the existing model structure; cannot compensate for unmodeled dynamics or structural model errors.

Residual modeling is a technique for bridging the domain gap by learning a secondary, data-driven model that predicts the error between a first-principles simulation and reality. This creates a grey-box hybrid model.

• How it works: A neural network is trained to predict the discrepancy (residual) in dynamics or observations. This network is then added to the simulator's outputs during training.
• Benefit: Allows policies to learn with the structured prior of physics and adapt to the systematic errors unique to the target hardware, effectively learning the domain gap.