Sim-to-Real Gap

Instead of training in a single, perfect simulation, this technique randomizes the visual and physical parameters of the simulator during training. By varying lighting conditions, textures, object masses, and friction coefficients across thousands of episodes, the policy learns to ignore irrelevant visual noise and focus on invariant task dynamics. This forces the model to generalize to reality without ever seeing real data, as the real world appears to the model as just another random seed in the distribution.

A process of building a highly accurate digital twin by measuring and calibrating the physical parameters of the real-world system. This involves characterizing actuator dynamics, joint backlash, sensor noise profiles, and latency distributions and encoding them directly into the simulator. By minimizing the physical parameter mismatch between the simulation and the specific robot hardware, the policy experiences a state transition function that closely mirrors reality, reducing the need for massive generalization.

A technique that aligns the feature representations of the simulated and real domains using unpaired or paired data. Methods like Generative Adversarial Networks (GANs) are used to translate synthetic images into photorealistic images (pixel-level adaptation), or adversarial loss functions are used to confuse a domain classifier so the policy extracts domain-invariant features. This bridges the visual gap by ensuring the policy's internal representation of a 'door handle' is identical whether it is rendered or captured by a real camera.

An architecture designed to prevent catastrophic forgetting during transfer. A deep neural network is first trained to mastery in simulation. When transferred to the real robot, a new column of layers is instantiated and connected via lateral connections to the frozen simulation layers. This allows the model to learn real-world residuals and fine-tune its policy without overwriting the robust visual features learned in simulation, effectively leveraging prior knowledge while adapting to the new dynamics.

A specific subset of domain randomization focused purely on the physics engine. Parameters such as gravity, damping coefficients, motor torque curves, and time-step sizes are heavily randomized during training. The policy learns to be robust to a wide range of physical dynamics, effectively treating the real-world physics as just another sample from the training distribution. This is particularly effective for contact-rich manipulation tasks where accurate friction modeling is impossible.

Trains a model not just to perform a task, but to learn how to adapt to a new task or environment quickly. The model is exposed to a distribution of simulated environments with varying dynamics. The optimization objective is to find parameters that can adapt to a new, unseen environment within a few gradient steps. When deployed on a physical robot, the model uses a small amount of real-world interaction data to rapidly fine-tune its internal dynamics model, bridging the gap in real-time.

A transfer learning approach that explicitly aligns the feature representations between the source (simulation) and target (real-world) domains. Unlike randomization, this often uses unlabeled real-world data to minimize the domain discrepancy through adversarial training or statistical moment matching.

• Uses techniques like Gradient Reversal Layers
• Aligns latent feature distributions
• Requires some real-world data for unsupervised alignment

The process of building a mathematical model of a physical system by measuring its real-world parameters and calibrating the simulator to match. This closes the gap by making the simulation physically accurate rather than making the policy robust to inaccuracies.

• Measures real friction coefficients, motor delays, and backlash
• Creates a digital twin with high-fidelity dynamics
• Common in industrial robotics and drone control

Often used interchangeably with Sim-to-Real Gap, this term specifically emphasizes the irreducible physical differences that no simulator can perfectly capture, such as complex contact dynamics, deformable objects, and unpredictable environmental interactions.

• Highlights the limits of rigid-body physics engines
• Drives research into differentiable simulators
• Acknowledges that some phenomena require real-world fine-tuning

A neural network architecture designed to prevent catastrophic forgetting during transfer. Lateral connections to previously learned features are frozen, while new columns are added for the target domain. This allows the policy to leverage simulation features while adapting to real-world nuances without overwriting prior knowledge.

• First introduced by DeepMind for continual learning
• Enables safe sim-to-real transfer without performance collapse
• Maintains a reusable library of simulation-trained features

A specific subset of domain randomization that focuses exclusively on physical parameters: mass, inertia, joint damping, friction, and actuator latency. By training a policy across an extreme range of dynamics, the agent learns robust control strategies that are insensitive to the true physical properties of the deployment platform.

• Critical for legged locomotion transfer
• Often paired with actuator network modeling
• Eliminates the need for precise system identification