Sensor Calibration

Intrinsic parameters model the internal geometry and optical characteristics of a sensor. For cameras, this includes:

• Focal length: Distance from the lens to the image sensor, determining the field of view.
• Principal point: The optical center's projection onto the image plane (cx, cy).
• Distortion coefficients: Radial (k1, k2, k3) and tangential (p1, p2) parameters that correct for lens aberrations, mapping distorted pixel coordinates to their ideal pinhole positions. For an IMU, intrinsic calibration involves determining scale factors, biases, and non-orthogonalities for each accelerometer and gyroscope axis.

Extrinsic parameters define the position and orientation (the 6-DOF pose) of one sensor relative to another or to a common reference frame (e.g., the robot's base link). Represented as a rigid transformation comprising a 3x3 rotation matrix R and a 3x1 translation vector t. Accurate extrinsic calibration is critical for tightly-coupled fusion algorithms like VIO and LIO, as misalignment introduces systematic errors that corrupt the fused state estimate. Calibration often uses a known target (e.g., a checkerboard) visible to multiple sensors.

Temporal calibration solves for the time offset (or latency) between the clocks of different sensors. A mis-synchronization of even a few milliseconds can cause significant error for a fast-moving robot. This involves determining the constant delay and potentially the clock drift between sensors. Techniques often involve observing a repetitive or correlated signal (e.g., a flashing LED, IMU excitation during motion) across sensors. Time synchronization protocols like PTP (Precision Time Protocol) are used in hardware, while software-based methods estimate residual offsets.

Calibration must characterize a sensor's stochastic noise properties, which are essential for probabilistic filters like the Kalman Filter. Key parameters include:

• Noise covariance matrices: Define the magnitude (variance) and correlation of measurement noise (R) and process noise (Q).
• Random Walk and Bias Instability: For IMUs, these parameters model how gyroscope and accelerometer biases evolve over time (e.g., specified as deg/√hr or m/s/√hr).
• Allan Variance: A time-domain analysis used to identify and quantify different noise sources (quantization, white noise, bias instability) in inertial sensors. These parameters directly influence the filter's gain and uncertainty estimates.

Beyond simple scale and bias, high-precision calibration accounts for nonlinearity in the sensor's response curve across its operational range. For IMUs and force-torque sensors, cross-axis sensitivity (or misalignment) is critical—it measures how much a stimulus along one axis produces an output on another orthogonal axis. This is represented by a 3x3 matrix (often near-identity) that corrects for non-orthogonal sensor axes relative to the package frame. Ignoring these effects leaves structured residuals that degrade estimation accuracy.

Calibration requires a known ground truth or excitation signal.

• Camera: Uses planar checkerboard or AprilGrid patterns to solve for intrinsics and lens distortion via Zhang's method or bundle adjustment.
• LiDAR-Camera: Uses patterns with known 3D geometry visible to both sensors to solve for extrinsics.
• IMU: Requires precise tumble or rotation stages to excite all axes, or uses motion capture for kinematic calibration.
• Multi-Sensor Suite: Often performed sequentially (intrinsics first) or jointly in a factor graph optimization that solves for all parameters simultaneously by minimizing reprojection and prediction errors.

The algorithmic process of combining data from multiple disparate sensors (e.g., camera, LiDAR, IMU) to produce a more accurate, complete, and reliable estimate of a system's state than is possible from any single sensor. Calibration is a prerequisite for effective fusion.

• Types: Loosely-coupled vs. tightly-coupled architectures.
• Goal: Overcome individual sensor limitations (noise, drift, occlusion).
• Output: A unified state estimate (pose, velocity, map) for navigation and control.

The process of inferring the internal, often unmeasured, state variables (e.g., position, orientation, velocity) of a dynamic system from a sequence of noisy sensor measurements. It is the core computational problem solved by fusion algorithms.

• Framework: Often formulated as a Bayesian filtering problem.
• Key Challenge: Separating true signal from sensor noise and systematic error (bias).
• Applications: Visual-Inertial Odometry (VIO), LiDAR SLAM, and target tracking.

The critical process of aligning the timestamps of data from different sensors to a common, precise clock. Without synchronization, even perfectly calibrated sensors cannot be fused correctly, as their measurements correspond to different moments in time.

• Hardware Methods: Using GPS PPS signals or dedicated synchronization chips.
• Software Methods: Post-hoc timestamp interpolation and alignment.
• Impact: Mis-synchronization directly translates to pose estimation errors, especially at high speeds.

A mathematical function that describes how sensor observations are generated from the true state of the system. It defines the expected sensor output for a given state, incorporating sensor geometry and noise characteristics determined during calibration.

• Components: Includes intrinsic parameters (lens distortion, focal length) and extrinsic parameters (sensor pose).
• Role in Estimation: Used in filters (e.g., Kalman Filter) to compute the difference between predicted and actual measurements (innovation).
• Example: A pinhole camera model projects a 3D point in the world to a 2D pixel coordinate.

A bipartite graphical model used to represent and solve large-scale state estimation problems by factorizing a complex probability distribution. Sensor calibration constraints and measurements are encoded as factors within this graph.

• Structure: Nodes represent state variables (poses, landmarks); factors represent constraints (odometry, sensor measurements, calibration priors).
• Solution: Maximum A Posteriori (MAP) estimation is performed via nonlinear optimization (e.g., Levenberg-Marquardt).
• Use Case: The standard backend for modern SLAM and bundle adjustment systems.

A specific state estimation technique that fuses data from a camera and an Inertial Measurement Unit (IMU) to estimate a robot's 3D pose and velocity. It critically depends on accurate spatiotemporal calibration between the camera and IMU.

• Principle: The camera provides absolute but intermittent measurements; the IMU provides high-frequency relative motion that bridges visual gaps.
• Challenge: Requires estimating and compensating for time offset and sensor bias in addition to extrinsic pose.
• Output: Drift-reduced, high-frequency ego-motion estimate essential for agile drones and AR/VR.