System Identification

System identification is fundamentally an empirical, data-driven process. Unlike first-principles modeling, which derives equations from physical laws, it infers a model's structure and parameters directly from observed system behavior. The core workflow involves:

• Applying known excitation signals (inputs) to the system.
• Recording the corresponding system response (outputs).
• Using statistical and optimization algorithms to find the model that best explains the observed input-output relationship.

This makes it essential for modeling complex, poorly understood, or highly nonlinear systems where analytical models are intractable.

A key distinction in system identification is the choice of model representation.

Parametric Models assume a predefined mathematical structure (e.g., state-space equations, transfer functions) with a finite set of unknown parameters to be estimated. Examples include ARX (AutoRegressive with eXogenous input) and OE (Output Error) models. They are compact and interpretable but require correct structural assumptions.

Non-Parametric Models describe the system without a fixed structure, often using impulse/step responses or frequency response functions. They are more flexible but can be less efficient for simulation and control design. The choice depends on the intended use of the model, such as controller design (parametric) or initial analysis (non-parametric).

The primary application in sim-to-real transfer is to reduce the reality gap. Even high-fidelity physics simulators contain inaccuracies in parameters like mass, inertia, friction coefficients, and actuator dynamics. System identification directly targets this by:

• Collecting input-output data from the physical robot (e.g., motor commands and resulting joint angles).
• Using this data to calibrate the simulation's physical parameters, minimizing the discrepancy between simulated and real behavior.
• This results in a digital twin—a high-fidelity virtual replica—that can be used for reliable policy training, safe testing, and hardware-in-the-loop validation before physical deployment.

System identification methodologies are categorized by the level of prior knowledge incorporated.

Black-Box Identification assumes no prior physical knowledge. The model is learned purely from data using flexible function approximators like neural networks. It's powerful for complex systems but offers little interpretability.

Grey-Box Identification incorporates partial known physics (e.g., the structure of dynamics equations) while estimating unknown parameters. This balances data efficiency with physical plausibility and is common in robotics.

White-Box Identification relies entirely on first-principles physics, using data only to fine-tune a small number of parameters (e.g., exact friction coefficient). It provides maximum interpretability but requires complete and accurate physical models.

The quality of an identified model is critically dependent on the excitation signal used during data collection. A poor input signal will fail to reveal the system's full dynamic behavior.

Persistence of Excitation (PE) is a formal condition requiring the input signal to be sufficiently rich and varied to excite all relevant modes of the system over time. Common excitation signals include:

• Pseudo-Random Binary Sequences (PRBS): Switch between levels to excite a broad frequency range.
• Chirp Signals: Sine waves with linearly increasing frequency.
• Step and Impulse Inputs: For simple, time-domain analysis.

Without persistent excitation, parameter estimates may be biased or non-unique, leading to an inaccurate model.

A final, critical step is model validation to ensure the identified model generalizes beyond the data used for estimation. This prevents overfitting—where the model memorizes noise in the training data.

Standard practices include:

• Cross-Validation: Estimating parameters on one dataset and testing prediction accuracy on a separate, unseen validation dataset.
• Residual Analysis: Checking if the prediction errors (residuals) are uncorrelated and resemble white noise, indicating all systematic behavior has been captured.
• Simulation vs. Prediction: Comparing the model's multi-step simulation output (using its own predictions as input) to real data, which is a stricter test than one-step prediction.

A model that fails validation must be re-estimated with a different structure or more informative data.

System identification is used to estimate the physical parameters of a robot or its environment to create a high-fidelity digital twin. This involves collecting data from real hardware—such as joint torques, positions, and velocities—and optimizing simulation parameters (e.g., mass, inertia, friction coefficients) to minimize the discrepancy between simulated and real behavior. Accurate calibration reduces the reality gap, making simulation a reliable proxy for real-world training and testing.

In model-based reinforcement learning (MBRL), an agent learns a dynamics model of its environment to plan optimal actions. System identification provides the methodology to learn this model from interaction data. A well-identified model allows for efficient sample-efficient learning in simulation and safer on-policy data collection on real robots, as the agent can predict outcomes before taking risky physical actions.

Controllers for robots often rely on an internal model of the system's dynamics. System identification enables adaptive control by continuously updating this model in response to changes, such as payload variations or wear and tear. It also informs robust control design by quantifying parameter uncertainties, allowing controllers to maintain stability and performance across a range of possible real-world conditions identified during modeling.

Effective domain randomization requires varying simulation parameters within plausible bounds. System identification provides the empirical data to define these bounds. For example, by measuring the actual friction coefficients across different floor types in a warehouse, engineers can set a randomization distribution (e.g., uniform between 0.2 and 0.6) that encompasses real-world variability, leading to more robust and generalizable policies.

By establishing a baseline model of normal system behavior, deviations in observed data can signal faults or degradation. System identification techniques are used to build these diagnostic models, enabling:

• Anomaly detection in sensor streams.
• Predictive maintenance by tracking how identified parameters (e.g., motor resistance) drift over time.
• Isolation of faulty components in complex robotic systems.

A primary goal in sim-to-real transfer is zero-shot deployment. High-quality system identification minimizes the need for costly and time-consuming real-world fine-tuning. By providing a simulation that closely matches the dynamics of the target hardware, policies trained in simulation achieve higher initial success rates upon physical deployment, accelerating the overall development and deployment lifecycle for robotic applications.

The reality gap, also known as the simulation-to-reality gap, is the performance discrepancy between a model trained in simulation and its performance when deployed on a physical system. This gap arises from modeling inaccuracies in the simulator, such as simplified physics, imperfect sensor models, or unmodeled actuator dynamics. System identification directly targets this gap by calibrating the simulation's parameters to better match the real world's behavior, thereby reducing the gap and improving zero-shot transfer success.

Physics randomization is a core domain randomization technique that specifically varies the parameters of a simulation's physics engine. Unlike system identification, which seeks a single accurate model, physics randomization intentionally creates a distribution of models by sampling parameters like mass, friction, inertia, and motor gains from a defined range. This trains policies to be robust across many possible physical realities. System identification can be used to define the plausible bounds for this randomization, ensuring the varied parameters stay within physically realistic limits.

Digital twin creation involves building a high-fidelity virtual replica of a specific physical asset or system for monitoring, analysis, and control. While related, it has a different focus than general system identification. A digital twin requires extremely accurate, instance-specific calibration—often using continuous data streams from the physical twin. In contrast, system identification for sim-to-real might aim for a "good enough" model that supports robust policy training, not necessarily a perfect one-to-one mirror. Both processes rely on similar parameter estimation techniques but serve different end goals: precise replication versus robust generalization.

In the context of system identification and domain randomization, the parameter space defines the complete set of simulation parameters that can be estimated or varied. Key categories include:

• Dynamic Parameters: Mass, center of mass, inertia tensors.
• Contact Parameters: Friction coefficients, restitution, contact stiffness.
• Actuator Parameters: Motor torque constants, gear ratios, backlash, latency.
• Sensor Parameters: Noise models (Gaussian, bias), latency, field of view distortion. System identification seeks to find the optimal point in this high-dimensional space that minimizes the error between simulated and real system behavior.

Hardware-in-the-loop testing is a validation methodology where physical robot hardware (e.g., actuators, sensors, or the full robot) is integrated in real-time with a simulation. This bridges system identification and final deployment. The identified simulation model runs the virtual environment, while commands are sent to and readings are taken from the real hardware. HIL allows for safe, accelerated testing of control policies under realistic dynamics before full physical deployment. It provides a critical feedback loop for refining both the system identification model and the policy itself.

The simulation fidelity trade-off is the engineering balance between computational cost and model accuracy. A high-fidelity simulator with complex contact models and fluid dynamics may be computationally prohibitive for the millions of trials needed in reinforcement learning. System identification often works with lower-fidelity, real-time simulators (e.g., using rigid body dynamics). The goal is not perfect photorealism or physics, but identifying the key parameters that most affect policy performance. This trade-off dictates which parameters are worth identifying precisely and which can be left approximated or randomized.