Sim-to-Real Transfer

A technique that randomizes the visual and physical parameters of the simulator during training to prevent the policy from overfitting to a single synthetic environment.

• Varies lighting, textures, friction, and object masses
• Forces the model to learn invariant features
• The real world appears as just another randomization seed
• Example: OpenAI used this to train a robotic hand to solve a Rubik's cube purely in simulation

The process of building mathematical models of physical dynamics from measured input-output data to calibrate the simulator against reality.

• Estimates parameters like motor constants, damping coefficients, and backlash
• Creates a digital twin that accurately mirrors the target hardware
• Reduces the sim-to-real gap at the source
• Example: Identifying the precise latency and torque curve of a servo motor before training a grasping policy

A family of techniques that align feature representations between the source (simulation) and target (real) domains using unlabeled real-world data.

• Uses adversarial training to learn domain-invariant features
• Minimizes the Maximum Mean Discrepancy (MMD) between domains
• Requires a small set of real-world observations, not full labels
• Example: Adapting a vision-based grasping network trained on rendered CAD images to work with real camera feeds

An architecture that transfers knowledge across a sequence of tasks by instantiating a new neural network column for each environment while maintaining lateral connections to previously learned features.

• Prevents catastrophic forgetting of simulation-learned skills
• Enables fine-tuning on the physical robot without losing prior knowledge
• Lateral connections allow the real-world column to reuse simulated features
• Example: Training first on a low-fidelity simulator, then a high-fidelity one, and finally on the physical robot

A specific form of domain randomization that perturbs the physical parameters of the simulation at the start of each training episode.

• Randomizes masses, joint friction, actuator gains, and contact parameters
• The policy learns a robust control strategy that works across a distribution of dynamics
• Does not require an accurate system ID model
• Example: Training a quadruped locomotion policy that walks on any surface by randomizing ground friction and restitution coefficients

A technique that maps observations from both simulation and reality into a shared latent representation where the policy can operate without distinguishing between domains.

• Uses autoencoders or contrastive learning to learn a unified embedding
• The policy operates entirely in the latent space, decoupled from raw sensor specifics
• Requires paired or unpaired sim-real image data
• Example: Embedding both synthetic and real camera images into a common feature space so a visual servoing policy transfers seamlessly

A foundational sim-to-real technique that varies the visual and physical parameters of the simulator non-physically during training.

• Randomizes lighting, textures, friction, and mass
• Forces the policy to see the simulator's visuals as irrelevant noise
• The real world becomes just another randomized instance
• Prevents overfitting to specific synthetic aesthetics

Example: An object detection model trained with randomized backgrounds and lighting can identify parts on a real factory floor despite never seeing the actual environment.

The process of building mathematical models from measured input-output data to close the sim-to-real gap.

• Used when first-principles physics models are unavailable or inaccurate
• Captures real friction, backlash, and actuator dynamics
• Creates a high-fidelity digital twin for training
• Often uses recursive least-squares or neural network estimators

Example: Recording a robot arm's actual joint torque response to commanded voltages creates a data-driven dynamics model that makes the simulator behave like the physical hardware.

A transfer learning approach that aligns feature distributions between simulated source data and real target data.

• Uses adversarial training to learn domain-invariant representations
• Minimizes Maximum Mean Discrepancy (MMD) between domains
• Can operate at the pixel level or feature level
• Requires a small set of unlabeled real-world samples

Example: A grasping network trained in simulation uses adversarial domain adaptation to make its visual features indistinguishable from those of real camera feeds, enabling zero-shot transfer.

A neural architecture that lateral connections to previously learned features while training on a new domain, preventing catastrophic forgetting.

• Each new domain adds a column to the network
• Lateral connections enable feature reuse without interference
• The simulation-trained column remains frozen and intact
• Enables sequential transfer across multiple simulators

Example: A locomotion policy first trained in a low-fidelity simulator transfers to a high-fidelity simulator, then to a real robot, with each stage adding a new column while preserving prior knowledge.

Quantitative measures that characterize the discrepancy between simulated and real trajectories or outcomes.

• Dynamic Time Warping (DTW) compares temporal sequences
• Frechet Inception Distance (FID) measures visual distribution gaps
• Mean Squared Error on state-action pairs
• Guides iterative simulator refinement

Example: Computing the DTW distance between a simulated and real bipedal walking gait reveals specific phases where the physics engine's ground contact model needs recalibration.

A digital twin architecture that fuses physics-based simulation with data-driven machine learning components to achieve higher accuracy than either approach alone.

• Physics engine handles known dynamics
• Neural network learns unmodeled residuals
• Corrects for simulator bias in real-time
• Enables continuous sim-to-real refinement

Example: A quadcopter simulator computes ideal aerodynamic forces while a learned residual model corrects for unmodeled ground-effect turbulence, enabling precise real-world landing maneuvers.