Ground Truth Alignment

This step aligns the timestamps of simulated and real-world data streams. It involves compensating for:

• Clock drift between simulation and sensor hardware
• Variable latencies in sensor processing pipelines
• Sampling rate mismatches through resampling or interpolation

Accurate synchronization is foundational for comparing dynamic behaviors like trajectories and forces, where millisecond misalignments can invalidate fidelity metrics.

This process establishes a common coordinate frame between the simulated environment and the physical world. Key tasks include:

• Marker-based calibration using fiducial markers (e.g., AprilTags, Charuco boards)
• Point cloud alignment via algorithms like Iterative Closest Point (ICP) for 3D data
• Extrinsic camera calibration to map simulation camera poses to real-world camera poses

Without precise registration, comparisons of position, orientation, and scene geometry are meaningless.

This involves mapping specific simulated sensor outputs to their real-world counterparts. For a robotic system, this means aligning:

• Joint encoders with simulated joint angles
• IMU readings with simulated angular velocity and linear acceleration
• Camera images with synthetic renderings from the same viewpoint
• Force/torque sensor data with simulated contact forces

Proper association ensures the ground truth data directly corresponds to the simulated variable being validated.

Before comparison, raw data streams must be processed into a comparable state. This includes:

• Applying identical noise models or filters to both streams (or removing noise from real data)
• Handling missing data and sensor dropouts consistently
• Converting units and coordinate conventions (e.g., quaternions to rotation matrices)
• Synchronized downsampling to a common, manageable frequency for analysis

This step minimizes artifacts not related to the underlying system dynamics.

Once aligned, quantitative fidelity metrics are calculated to measure the discrepancy. Common metrics include:

• Root Mean Square Error (RMSE) for trajectory tracking
• Structural Similarity Index (SSIM) for visual renderings
• Peak Signal-to-Noise Ratio (PSNR) for image-based comparisons
• Dynamic Time Warping (DTW) distance for time-series with slight temporal distortions

These metrics feed directly into the parameter calibration and system identification feedback loop.

The final step is a closed-loop process where alignment quality is iteratively assessed and improved:

1. Initial alignment is performed.
2. Residual errors are analyzed to diagnose causes (e.g., poor calibration, unmodeled dynamics).
3. Simulation parameters (e.g., camera intrinsics, timing offsets) are adjusted.
4. The alignment process is re-run.

This loop continues until error metrics fall below a predefined threshold, ensuring the simulation is a valid digital twin for the purpose of training or testing.

System identification is the process of constructing mathematical models of dynamic systems from measured input-output data. It is the core methodology that ground truth alignment enables.

• Purpose: To characterize a system's behavior and estimate its unknown physics parameters (e.g., mass, friction, inertia).
• Process: Involves designing excitation trajectories, collecting data, selecting a model structure, and performing parameter estimation.
• Types: Includes grey-box identification (combining physics and data) and Bayesian calibration (probabilistic parameter inference).

Model fidelity is the degree to which a simulation's outputs match the real-world system it replicates. It is the ultimate goal improved by ground truth alignment.

• Assessment: Measured using quantitative validation against aligned ground truth data.
• Metrics: Evaluated with fidelity metrics like mean squared error (MSE) on state trajectories or task-specific success rates.
• Limitations: Constrained by simulation bias (systematic simulator error) and unmodeled dynamics (phenomena not captured by the model).

Parameter calibration is the iterative adjustment of a simulation's numerical constants to minimize discrepancy with real-world observations. It is the primary action taken after ground truth alignment.

• Inputs: Uses temporally aligned data streams from simulation and physical sensors.
• Methods: Ranges from optimization-based data-driven calibration to probabilistic Bayesian calibration.
• Output: A set of tuned physics parameters (e.g., coefficients of friction, motor gains) that reduce calibration error.

The reality gap is the performance drop observed when a policy trained in simulation is deployed on a physical robot. Ground truth alignment helps diagnose and reduce this gap.

• Cause: Primarily driven by the domain gap—the statistical difference between simulated and real data distributions.
• Manifestation: Measured as transfer error (e.g., increased task failure rate, higher energy consumption).
• Bridging: Addressed by improving model fidelity through calibration and using techniques like domain randomization.

Quantitative validation is the rigorous, metrics-based process of assessing a simulation's accuracy. It is impossible without first performing ground truth alignment.

• Procedure: Compare numerical outputs (torques, positions, velocities) from the simulator against high-accuracy, time-synchronized physical measurements.
• Tools: Employs statistical fidelity metrics like RMSE, Pearson correlation, or maximum absolute error.
• Role: Forms the final step in a system ID pipeline, providing an objective measure of model usefulness for training or testing.

Residual modeling is a technique to compensate for imperfections in a physics-based simulator. It uses the error signal revealed by ground truth alignment.

• Concept: A secondary, often data-driven model (e.g., a neural network) is trained to predict the discrepancy (residual) between the first-principles simulator and real-world data.
• Application: The residual model's predictions are added to the base simulator's outputs, effectively learning the unmodeled dynamics or model uncertainty.
• Advantage: Creates a higher-fidelity hybrid model without requiring a complete rewrite of the core physics engine.