Residual Modeling

Residual modeling creates a compound model where a primary first-principles simulator (e.g., based on rigid-body dynamics) provides a baseline prediction. A secondary data-driven model (typically a neural network) is then trained to predict the residual error—the discrepancy between the simulator's output and real-world sensor data. This architecture explicitly separates known physics from learned corrections.

• White-Box Core: The physics simulator encodes domain knowledge and generalizes to unseen states.
• Black-Box Corrector: The residual model captures unmodeled dynamics, complex friction, actuator nonlinearities, or aerodynamic effects that are difficult to derive analytically.

The primary function of a residual model is to compensate for unmodeled dynamics—physical phenomena absent from the simplified equations of the base simulator. Common sources of error it learns include:

• Complex Contact Dynamics: Soft-body deformations, rolling friction, and non-ideal collision responses.
• Actuator Saturation and Backlash: Non-linearities in motors, gears, and hydraulic systems.
• Fluid and Aerodynamic Effects: Drag, lift, and turbulence that are computationally expensive to simulate in real-time.
• Sensor Noise and Bias: Systematic errors in measurement models (e.g., camera distortion, IMU drift). By learning these residuals, the overall simulation achieves a higher effective fidelity without the computational cost of modeling every physical detail from first principles.

Residual models are trained via data-driven calibration. The process involves:

1. Collect Real-World Trajectory Data: Execute excitation trajectories on the physical robot to gather state-action pairs (s_t, a_t, s_{t+1}).
2. Generate Paired Simulation Rollouts: For each real (s_t, a_t), run the base simulator to get its predicted next state s_{t+1}^{sim}.
3. Compute the Residual: The training target is the difference: δ = s_{t+1}^{real} - s_{t+1}^{sim}.
4. Train the Model: A neural network f_θ(s_t, a_t) is trained to predict δ, minimizing a loss like Mean Squared Error (MSE). This is a form of grey-box identification, blending physics-based structure with data-driven refinement.

The core objective is to minimize the reality gap—the performance drop when a policy trained in simulation transfers to reality. By making the simulator's dynamics more domain-aligned with the target environment, residual modeling directly addresses the domain gap.

• Policy Robustness: Policies trained in a residual-augmented simulator encounter dynamics closer to reality, making them more robust upon deployment.
• Reduced Transfer Error: This leads to lower transfer error metrics, such as higher task success rates and reduced need for policy adaptation or fine-tuning on the physical system. It acts as a targeted bridge, narrowing the statistical difference between the simulation and real-world data distributions.

Residual modeling is a critical enabler for Sim-to-Real Transfer Learning. It allows for the creation of a high-fidelity digital twin where reinforcement learning agents can be trained safely and at scale.

• Safe, Parallelized Training: Dangerous or expensive-to-repeat failures happen in simulation.
• Accelerated Development: Training in simulation is orders of magnitude faster than real-time. Residual models make this training more relevant.
• Foundation for Domain Randomization: The corrected simulator can serve as a more accurate base for domain randomization, where parameters are varied within physically plausible bounds derived from the residual model's learned uncertainty.

Residual modeling is a specialized form of grey-box system identification. While traditional parameter estimation seeks to find point values for physics parameters (e.g., mass, friction), residual modeling often assumes those are insufficient and learns a flexible error function.

• Complement to Calibration: It can be used after parameter calibration to mop up remaining calibration error.
• Handles Model Uncertainty: It directly addresses model uncertainty by learning a distribution over possible errors.
• Dynamic Regressor Alternative: Unlike a dynamic regressor which linearly maps to physical parameters, a neural residual model can capture highly non-linear, state-dependent errors that a linear model cannot.

The foundational process of constructing mathematical models of a dynamic system from measured input-output data. It focuses on estimating the physics parameters (e.g., mass, friction) of a first-principles model. Residual modeling often builds on top of a system identification baseline to capture what the physics model misses.

• Core Goal: Find the parameters of a known model structure (e.g., Lagrangian dynamics).
• Contrast with Residual Modeling: System ID seeks explainable parameters; residual modeling learns an unstructured error correction.

The specific physical phenomena or complex behaviors that are not captured by the first-principles simulation model. This is the explicit target of a residual model.

• Examples: Stiction in gears, cable elasticity, complex aerodynamic drag, hysteresis in actuators, and wear-and-tear effects.
• Impact: Unmodeled dynamics are the primary source of the reality gap, causing controllers trained in simulation to fail on real hardware. The residual model's job is to predict and compensate for these effects.

A hybrid modeling paradigm that combines a white-box (physics-based) model with a black-box (data-driven) model. Residual modeling is a canonical example of grey-box identification.

• Structure: Full Model = Physics Model(θ) + Data-Driven Residual Network(φ)
• Advantage: Leverages physical intuition for generalizability while using data to capture intractable complexities. This is more data-efficient and interpretable than a purely black-box neural network learning dynamics from scratch.

The performance discrepancy observed when a policy or model trained in a simulation is deployed on the real physical system. Residual modeling is a direct technique for bridging this gap.

• Primary Causes: Unmodeled dynamics, sensor noise, actuation latency, and simulation bias.
• Mitigation Strategy: By learning the error between simulation and reality, the residual model effectively creates a corrected simulator. Policies trained in this corrected environment exhibit significantly lower transfer error.

The process of adjusting the numerical values of a simulation's physics parameters to minimize discrepancy with real-world data. It is often a prerequisite step before residual modeling.

• Process: Uses optimization or system ID techniques to tune parameters like mass, inertia, and friction coefficients.
• Relationship to Residual Modeling: First, calibrate the base physics model as well as possible. Then, use residual modeling to capture whatever systematic error remains after calibration, which is attributed to unmodeled dynamics.

A deliberately designed sequence of robot motions or control inputs that is rich enough to excite all the dynamic modes necessary for high-quality system identification or residual model training.

• Purpose: To collect data that reveals the full range of the system's dynamics, including nonlinear and unmodeled effects.
• Property: Must provide persistent excitation.
• Design: Often involves sweeping frequencies, maximizing accelerations, and exploring joint limits to ensure the residual network learns a comprehensive error model.