Sim2Real Gap

A robot trained in simulation to pick up a specific object often fails when deployed because of unmodeled physical properties in the real world. Key discrepancies include:

• Surface friction and texture: Simulated grippers may assume perfect friction, while real objects can be slippery, deformable, or sticky.
• Object mass and inertia: Incorrect mass distribution in the simulator leads to inaccurate force predictions, causing drops or overshoots.
• Sensor noise and latency: Perfect camera feeds in simulation versus real-world issues like motion blur, variable lighting, and sensor delay. This gap necessitates techniques like domain randomization and real-world fine-tuning to achieve reliable performance.

The perception stack of a self-driving car—trained on synthetic data—faces a severe sim2real gap in visual recognition. Critical failures arise from:

• Non-photorealistic rendering: Simulated trees, buildings, and road signs lack the visual noise, weathering, and infinite variety of the real world.
• Weather and lighting conditions: Simulating the precise physics of rain on a windshield, snow accumulation, or harsh shadows is extremely difficult.
• Rare and adversarial scenarios: Simulators struggle to generate the long tail of edge cases, like a plastic bag blowing across the road or a pedestrian wearing unusual clothing. Bridging this gap requires massive domain adaptation and training on carefully curated real-world datasets.

Autonomous Mobile Robots (AMRs) for logistics are often first trained and validated in high-fidelity simulations of warehouses. The sim2real gap manifests in:

• Floor friction and wheel slip: Simulated perfect traction versus real polished concrete, spills, or uneven surfaces.
• Dynamic obstacle prediction: Simulated humans and other robots follow simplified rules, while real agents exhibit complex, unpredictable social navigation.
• Communication latency and packet loss: Network conditions between robots and a central orchestrator are idealized in simulation. Companies deploy digital twins that are continuously updated with real sensor data to narrow this gap and enable safe, efficient fleet orchestration.

Drones trained in simulation to navigate through windows or between trees frequently crash upon real deployment due to aerodynamic and sensor modeling errors:

• Unsteady aerodynamics: Simulators use simplified models that don't capture complex rotor wash, wind gusts, or ground effect.
• Inertial Measurement Unit (IMU) biases: Real IMUs have drift and calibration errors not present in simulation.
• Camera-radar-LIDAR misalignment: The precise extrinsic calibration between sensors is assumed perfect in sim, but small misalignments in the real system corrupt fusion algorithms. System identification and robust control policies (like those trained with domain randomization) are essential to overcome these discrepancies.

Precise tasks like assembly, welding, or painting require millimeter accuracy. The sim2real gap for robotic arms involves:

• Kinematic and dynamic calibration errors: Small inaccuracies in the robot's Denavit–Hartenberg parameters or joint friction models compound over long trajectories.
• Tool deformation and wear: Simulated end-effectors are rigid, while real tools bend, vibrate, and degrade over time.
• Contact dynamics: Simulating the precise forces during part insertion or surface contact is notoriously difficult, leading to jamming or damage. Adaptive control and online model parameter estimation are critical techniques used to compensate for these gaps in manufacturing settings.

Teaching a quadruped or biped robot to walk via simulation is a classic sim2real challenge. The gap is dominated by terrain and actuator modeling:

• Terrain compliance and granularity: Simulated ground is often uniformly hard, while real grass, mud, gravel, and carpets have complex, variable compliance.
• Actuator dynamics and backlash: High-fidelity modeling of electric motor torque curves, gearbox backlash, and hydraulic system response is computationally expensive and often simplified.
• Battery discharge effects: Real motor performance degrades as battery voltage drops, a factor rarely modeled. Breakthroughs like reinforcement learning with domain randomization (e.g., for Boston Dynamics' Spot or ANYmal) have shown that robust policies can emerge from training across a vast distribution of simulated physics parameters.

A core technique for Sim2Real transfer where a wide range of parameters in the simulator are randomly varied during training. This includes:

• Visual properties: lighting, textures, colors, camera angles.
• Physical dynamics: friction coefficients, object masses, actuator delays.
• Environmental factors: number of objects, background clutter. The goal is to train a policy that is robust to this variability, forcing it to learn the underlying task rather than overfitting to specific simulator quirks. It is a form of data augmentation at the simulation level.

The process of building or calibrating a simulator's parameters to match the dynamics of a specific real-world system. This involves:

• Collecting real-world data: Recording state-action trajectories from the physical robot.
• Parameter estimation: Using optimization (e.g., gradient descent, Bayesian optimization) to adjust simulator parameters (e.g., motor gains, link masses, friction) so that simulated rollouts match the real data.
• Iterative refinement: Often performed in a loop with policy training. It is a direct, engineering-focused approach to reducing the dynamics gap.

A class of machine learning techniques that adapt a model trained on a source domain (simulation) to perform well on a target domain (reality) with different data distributions. Key approaches include:

• Feature-level adaptation: Learning domain-invariant representations using adversarial training or discrepancy minimization.
• Pixel-level adaptation: Using techniques like CycleGAN to translate simulated images to look photorealistic.
• Fine-tuning: Limited real-world data is used to adjust the final layers of a policy or perception network pre-trained in simulation. This is distinct from Domain Randomization, which modifies the source domain itself.

A paradigm where physics simulators are implemented using differentiable operations, allowing gradients to flow from a loss (e.g., a task objective) back through the simulation steps. This enables:

• Direct policy optimization: Gradients can be backpropagated through the physics engine to the policy parameters.
• System identification: The simulator's physical parameters can be tuned by minimizing the difference between simulated and real trajectories.
• Bridging the gap: By making the simulator a differentiable component of the training loop, it can be directly adapted to better match real-world data, providing a powerful tool for model-based Sim2Real transfer.

A foundational paradigm for addressing Sim2Real. In MBRL, an agent learns an explicit world model (a dynamics and reward function) from experience. This model is then used for planning (e.g., via Model-Predictive Control) or to generate synthetic experience for policy training. The connection to Sim2Real is twofold:

1. The simulator is a world model: Training in simulation is a form of MBRL with a hand-crafted, potentially inaccurate model.
2. Learning to close the gap: Advanced MBRL agents like Dreamer learn their own world model from pixels; transferring such an agent involves adapting this learned model to real data, which is a model adaptation problem.