Unmodeled Dynamics

Unmodeled dynamics are discrepancies between a system's true physical behavior and the predictions of its mathematical model. They represent phenomena deliberately omitted for simplicity or unknowingly excluded due to incomplete knowledge.

• Inherent to All Models: All models are approximations; unmodeled dynamics are the residual error between this approximation and reality.
• Distinct from Parameter Error: Unlike inaccurate physics parameters (e.g., wrong mass value), unmodeled dynamics involve missing terms or structures from the model's equations entirely.
• Source of the Reality Gap: They are a primary contributor to the reality gap, causing policies trained in simulation to fail when deployed on physical hardware.

Unmodeled dynamics arise from specific, often complex, physical effects that are difficult to model precisely or are computationally prohibitive to simulate in real-time.

• Nonlinear Friction & Stiction: Static friction (stiction), viscous damping, and nonlinear stick-slip effects in joints and actuators.
• Actuator Dynamics: Delays, saturation, backlash, and non-ideal torque-speed curves in motors and gearboxes.
• Structural Flexibility: Bending and vibration in links, cables, or frames assumed to be perfectly rigid.
• Aerodynamic & Hydrodynamic Forces: Air resistance, lift, and turbulent flows on moving parts.
• Contact Dynamics: Complex, multi-point contact, rolling friction, and deformation during collisions.
• Thermal Effects: Expansion, contraction, and performance changes in electronics and materials.

The presence of unmodeled dynamics directly degrades the performance and reliability of both simulated training and real-world control systems.

• Degraded Controller Performance: Causes steady-state error, oscillations, or instability in feedback control loops, as the controller's internal model is incorrect.
• Poor Sim-to-Real Transfer: Policies optimized in an inaccurate simulation exploit its simplified dynamics, leading to failure on real hardware where those assumptions break down. This manifests as high transfer error.
• Challenges for System Identification: Complicates parameter estimation and grey-box identification, as the model structure itself is insufficient to fit the observed data.
• Increased Model Uncertainty: Contributes directly to overall model uncertainty, making robust or adaptive control strategies necessary.

Unmodeled dynamics are the inverse of model fidelity. A high-fidelity model has minimal unmodeled dynamics, while a low-fidelity model has significant omissions.

• Fidelity Spectrum: Models exist on a spectrum from low-fidelity (fast, simple) to high-fidelity (slow, complex). The choice involves a trade-off between computational cost and unmodeled dynamics.
• Quantitative Validation: Fidelity metrics (e.g., trajectory MSE) measure the aggregate effect of unmodeled dynamics by comparing simulation outputs to ground truth alignment data.
• Simulation Bias: Unmodeled dynamics introduce a simulation bias—a consistent, non-random deviation from reality—which is more problematic than simple noise for policy transfer.

Engineers use specific techniques to manage the risk posed by unmodeled dynamics, focusing on robustness, adaptation, and residual learning.

• Domain Randomization: Deliberately varying simulated physics parameters (e.g., friction ranges) during training to force the policy to become robust to a wide variety of dynamics, including unmodeled ones.
• Robust Control Design: Using control theories like H-infinity or sliding mode control that are inherently tolerant to bounded model errors.
• Residual Modeling: Training a secondary data-driven model (e.g., a neural network) to predict the discrepancy between the physics model and real sensor data, then compensating for it.
• Online Adaptation: Employing adaptive control or Bayesian calibration techniques that allow the system to estimate and compensate for dynamic errors in real-time.
• Excitation for Identification: Using excitation trajectories with persistent excitation during a system ID pipeline to illuminate and help characterize dynamic behaviors that simpler motions might miss.

Concrete instances highlight how unmodeled dynamics manifest in real engineering challenges.

• Quadrotor Drones: Unmodeled aerodynamic ground effect (cushion of air) causes instability during autonomous landing.
• Robotic Arms: Unmodeled cable dynamics (swinging power/encoder cables) introduce oscillatory disturbances not present in rigid-body forward dynamics simulations.
• Legged Robots: Unmodeled compliance and damping in foot pads or joints lead to inaccurate contact dynamics, causing slips or falls.
• Autonomous Vehicles: Unmodeled tire relaxation dynamics (the lag between slip angle and lateral force generation) degrade the accuracy of path-following controllers.
• Industrial Manipulators: Actuator dynamics like brushless DC motor commutation ripple create high-frequency torque vibrations not captured in standard models.

Joint friction is the resistance to motion in a robot's actuators and transmissions, often modeled simplistically as viscous friction. The more complex stiction (static friction) and Coulomb friction are frequently unmodeled. This leads to:

• Stick-slip motion during slow, precise movements.
• Torque prediction errors when the joint changes direction.
• Positional drift in gravity-compensated holds, as the controller underestimates the force needed to overcome initial stiction.

Actuator saturation occurs when a motor's torque, velocity, or current limits are reached, causing non-linear clipping not captured in linear models. Gearbox backlash is the dead zone between gear teeth when direction reverses.

Real-world impact: A controller commanding an acceleration based on an ideal model will not account for the delayed response from backlash or the reduced torque available at saturation, causing overshoot and oscillation, especially in high-speed, high-precision tasks like pick-and-place.

Many control models assume robot links and joints are perfectly rigid. In reality, structural flexibility in long links, harmonic drives, or cables introduces bending and torsional vibrations.

Key effects include:

• Low-frequency resonances that destabilize high-gain controllers.
• Oscillatory end-effector error after a rapid move.
• Non-collocation problems where the actuator sensor and end-effector are on different sides of a flexible element, making accurate state estimation difficult. This is critical in lightweight cobots and space manipulators.

While a robot's base dynamics may be identified, the inertia, center of mass, and grip stiffness of a payload are often unknown and variable. An unmodeled payload causes significant error in the required joint torques via the inverse dynamics calculation.

Consequences:

• Trajectory tracking error and vibration when moving an uncharacterized object.
• Degraded force control during insertion or assembly tasks.
• The need for rapid online adaptation in applications like warehouse logistics where items change constantly.

For robots operating in fluid environments—such as drones, underwater vehicles, or even high-speed arms—drag, lift, and vortex shedding forces are complex functions of velocity, shape, and fluid density. These are rarely included in standard rigid-body dynamics models.

Operational impact:

• Quadrotors experience unmodeled lateral drag during fast forward flight, causing positional error.
• Underwater manipulators have highly non-linear, velocity-dependent damping, leading to poor trajectory tracking if ignored.
• Energy consumption predictions become highly inaccurate.

Motor winding resistance increases with temperature, reducing available torque for the same current command. Battery voltage sag under high load reduces motor speed constants. Encoder quantization and electrical noise in sensor readings create unmodeled disturbances in the feedback loop.

These effects cause time-varying performance degradation:

• A robot may perform precisely when cold but become sluggish or inaccurate after extended operation as motors heat up.
• Battery state-of-charge effectively changes the plant model, a major concern for mobile robots and electric vehicles.

System identification is the foundational process of constructing mathematical models of dynamic systems from measured input-output data. It is the primary methodology for discovering and formalizing the dynamics that were previously unmodeled.

• Core Objective: To estimate unknown model parameters (like inertia, friction) or even the model structure itself from experimental data.
• Contrast with Unmodeled Dynamics: System ID is the active process; unmodeled dynamics are the residual phenomena that the process fails to capture.
• Common Techniques: Include grey-box identification, parameter estimation, and the use of excitation trajectories to ensure persistent excitation.

Model uncertainty is a broad, quantitative measure of the lack of perfect knowledge about a system's true dynamics. Unmodeled dynamics are a primary source of this uncertainty.

• Epistemic vs. Aleatoric: Unmodeled dynamics contribute to epistemic uncertainty (reducible through better modeling), as opposed to aleatoric uncertainty (inherent sensor noise).
• Quantification: Often represented probabilistically (e.g., confidence intervals, Bayesian posterior distributions) to inform robust control or safe exploration.
• Propagation: The impact of unmodeled dynamics, framed as uncertainty, can be propagated through simulations to predict potential performance degradation.

The reality gap is the observed performance discrepancy when a policy trained in simulation is deployed on a real system. Unmodeled dynamics are a direct, physical cause of this gap.

• Manifestation: Results in transfer error, such as a robot dropping an object or falling over due to unpredicted friction or compliance.
• Causal Relationship: Unmodeled dynamics create a domain gap between the simulated and real data distributions.
• Mitigation: Techniques like domain randomization and residual modeling are explicitly designed to bridge the gap caused by unmodeled or inaccurately modeled physics.

Residual modeling is a direct engineering response to unmodeled dynamics. It involves training a secondary, data-driven model to predict and compensate for the error between a physics-based simulator and reality.

• Methodology: The primary simulator (e.g., a rigid-body dynamics engine) runs, and a neural network learns to predict the discrepancy in observed states or forces.
• Hybrid Approach: Creates a grey-box model where first-principles physics handle known dynamics, and the data-driven component handles the residuals.
• Application: Critical for high-precision tasks where even small unmodeled effects (e.g., cable drag, motor hysteresis) cause significant calibration error.

Simulation bias is a systematic error introduced by the approximations and numerical methods of a simulator. Unmodeled dynamics are a form of structural bias, where entire physical phenomena are absent from the model.

• Sources: Includes discrete timestepping approximations, simplified contact models, ignored aerodynamic effects, and assumed perfect sensors.
• Distinction from Noise: Bias causes consistent deviation in one direction (e.g., simulated objects always slide farther), unlike random noise.
• Impact on Learning: Reinforcement learning policies can exploit simulation biases, learning behaviors that are infeasible in the real world, widening the reality gap.

Parameter calibration is the process of tuning a simulation's physics parameters (e.g., mass, friction coefficient) to align its output with real-world data. It addresses inaccurately modeled dynamics, whereas unmodeled dynamics lack any corresponding parameter.

• Goal: Minimize calibration error (e.g., mean squared trajectory error).
• Techniques: Ranges from manual tuning to data-driven calibration and Bayesian calibration.
• Limitation: Calibration can only adjust existing model parameters. It cannot introduce new dynamic terms to account for truly unmodeled phenomena, which is the domain of residual modeling or model augmentation.