Dynamics Mismatch

Simulations often use simplified contact models that fail to capture the complex, non-linear nature of real-world interactions.

• Penalty-based vs. Constraint-based methods: Many real-time simulators use fast, approximate penalty methods that allow unrealistic object interpenetration, unlike precise but computationally expensive constraint-based solvers.
• Coulomb friction approximations: Real friction involves stiction, viscous effects, and surface-dependent micro-slip, which are often reduced to a single static/dynamic coefficient pair in simulation.
• Example: A robot gripper may successfully lift a simulated box due to simplified friction, but fail in reality because the simulated friction coefficient does not account for surface texture or material compliance.

The simulated idealization of motors and transmissions ignores critical real-world latency, saturation, and backlash.

• Ideal torque control vs. real bandwidth: Simulations often assume actuators can deliver commanded torques instantaneously, ignoring the delay and filtering effects of motor drivers, gearboxes, and current loops.
• Saturation and rate limits: Real actuators have hard limits on maximum torque, velocity, and acceleration that are frequently unmodeled or overly simplified.
• Backlash and hysteresis: Mechanical play in gears and linkages creates non-linear, history-dependent behavior that is rarely simulated, leading to poor positioning accuracy when policies are transferred.

Even small discrepancies in an object's inertial properties—mass, center of mass, and moment of inertia tensor—can cause catastrophic failure in dynamic tasks.

• CAD model inaccuracies: Mass properties derived from CAD models assume uniform density and perfect geometry, ignoring internal components, wiring, and manufacturing variances.
• Payload uncertainty: For manipulation tasks, the mass and inertia of payloads are often unknown or variable, but are typically fixed in simulation.
• Impact on control: A policy trained to swing a simulated arm with an idealized inertia will generate incorrect joint torques for the real arm, causing overshoot, oscillation, or instability.

Simulations typically provide noiseless, full-state observations with zero latency, while real sensors introduce stochastic noise, bias, and processing delays.

• Proprioceptive noise: Joint encoders and IMUs have quantization error, electrical noise, and temperature-dependent bias.
• Perceptual latency: Camera images have exposure time, readout time, and network/processing latency, creating a mismatch between the simulated 'present' state and the real, slightly delayed observation.
• Policy impact: A policy relying on precise, instantaneous state feedback in simulation may become unstable when faced with the noisy, delayed feedback of real sensors.

Simulations often treat robots and objects as perfectly rigid bodies, ignoring the flexibility inherent in all real materials.

• Structural compliance: Robot links, gears, and mounting brackets bend under load, creating deflections not present in a rigid-body simulation.
• Contact compliance: Real contact involves deformation of both the robot end-effector and the environment (e.g., a soft gripper or a compressible object).
• Cable and hose dynamics: The mass and stiffness of cables supplying power and data to joints create drag forces and can snag, effects almost never modeled in standard simulators.

Controlled simulation environments lack the myriad of small, stochastic forces present in the real world.

• Air currents and thermal effects: Drafts from HVAC systems or heat from motors can affect sensor readings and create small forces on lightweight manipulators.
• Vibrations: Ground vibrations from other machinery or foot traffic are transmitted through the robot's base.
• Electrical noise: EMI/RFI from other equipment can induce noise in sensor and control signals.
• Consequence: A policy trained in a perfectly static, deterministic simulation has never learned to reject these persistent low-level disturbances, leading to steady-state error or jitter.

The reality gap, or sim2real gap, is the broad performance discrepancy between a policy's behavior in simulation and its behavior on physical hardware. Dynamics mismatch is a specific, critical subtype of this gap, focusing on inaccuracies in the physics model (e.g., friction coefficients, motor backlash, contact stiffness) rather than visual or observational differences.

System identification (SysID) is the process of building or refining a mathematical model of a physical system by analyzing its input-output data. It is the primary engineering method for reducing dynamics mismatch. Techniques include:

• Excitation Trajectories: Executing specific motor commands to observe the robot's response.
• Parameter Estimation: Using optimization (e.g., gradient descent) to find simulation parameters (mass, inertia, damping) that minimize the error between simulated and real sensor data.
• Grey-box Modeling: Combining known physics equations with data-driven correction terms.

Simulation bias refers to the systematic errors or simplifications inherent in a simulation engine that cause the training environment to differ from reality. Dynamics mismatch is a direct consequence of physics model bias. Common sources include:

• Contact Model Simplifications: Using spring-damper models instead of continuous collision detection.
• Actuator Modeling: Ignoring saturation, bandwidth limits, and non-linear torque-speed curves of real motors.
• Numerical Integration Error: Large timesteps or solver inaccuracies accumulating over long rollouts.

Domain randomization is a proactive sim-to-real technique that trains a policy in a simulation where environmental and system parameters are randomly varied within plausible bounds. For dynamics mismatch, this involves randomizing physics parameters such as:

• Link masses and inertias
• Joint friction and damping coefficients
• Motor force/torque limits
• Gravity vector By exposing the policy to a vast distribution of possible dynamics during training, it learns a robust control strategy that generalizes to the unmodeled dynamics of the real world, effectively 'averaging over' the mismatch.

Online adaptation refers to a policy's ability to adjust its parameters or behavior in real-time based on streaming data from the environment during execution. This is a critical reactive strategy for coping with residual dynamics mismatch after deployment. Methods include:

• Meta-Learning: Training a policy with a recurrent neural network or context encoder that can infer and compensate for unseen dynamics from recent experience.
• Adaptive Control: Using classical techniques like Model Reference Adaptive Control (MRAC) to update a low-level controller's parameters online.
• Bayesian Filtering: Continuously estimating hidden dynamics parameters as part of the state.

Uncertainty quantification (UQ) involves estimating the confidence or error bounds of a model's predictions. In the context of dynamics mismatch, UQ is used to identify out-of-distribution states where the simulation model is likely inaccurate and the policy's actions are unreliable. Techniques include:

• Ensemble Dynamics Models: Training multiple forward dynamics models; disagreement (variance) indicates epistemic uncertainty.
• Probabilistic Networks: Outputting a distribution over next states (e.g., using Gaussian outputs).
• Calibration Error: Measuring if a model's predicted confidence matches its actual frequency of being correct. High UQ signals can trigger safe fallback behaviors or request human intervention.