Sensor Calibration

Intrinsic parameters define the internal geometry and optical characteristics of a sensor. For cameras, this includes:

• Focal length: The distance between the lens and the image sensor, determining the field of view.
• Principal point: The optical center of the image, where the optical axis intersects the sensor plane.
• Lens distortion coefficients: Radial and tangential distortion parameters (e.g., k1, k2, p1, p2) that correct for the bending of light rays, especially in wide-angle lenses. For an IMU, intrinsic calibration involves determining the scale factors, biases, and non-orthogonalities of the accelerometer and gyroscope axes.

Extrinsic parameters define the position and orientation (the pose) of a sensor relative to a chosen reference frame, such as the robot's base link or another sensor. This is represented by a rotation matrix (R) and a translation vector (t).

• Calibrating extrinsics is critical for sensor fusion. For example, knowing the precise transform between a LiDAR and a camera allows point clouds to be projected onto images.
• The process often involves capturing data of a known calibration target (like a checkerboard) from multiple sensors simultaneously and solving for the optimal rigid transformation.

Temporal calibration synchronizes the timestamps of data streams from different sensors. Even small misalignments (< 100ms) can cause significant errors in high-speed applications like autonomous driving.

• This involves determining the time offset (latency) and potential clock drift between sensor clocks.
• Methods include recording simultaneous, easily detectable events (like an LED flash visible to a camera and a spike in an IMU) or using cross-correlation of signals from sensors observing the same dynamic process.

Real sensors exhibit complex, non-linear errors that go beyond simple Gaussian noise. Calibration aims to characterize these systematic errors.

• Bias: A constant offset added to all measurements (e.g., IMU bias).
• Scale factor error: A multiplicative error where the sensor's output is not linearly proportional to the true input.
• Cross-axis sensitivity: Signal from one axis leaking into another due to imperfect mechanical alignment.
• Temperature drift: Variation of bias and scale factor with operating temperature.
• Hysteresis: The sensor's output depends on the history of its inputs.

Calibration requires known, precise physical references. Common targets include:

• Checkerboard/Charuco board: Used for camera and camera-LiDAR calibration. The known size and pattern of squares provide correspondences between 3D points and 2D pixels.
• AprilTag: A fiducial marker system that provides a unique ID and a full 6-DOF pose, robust to occlusion and lighting.
• Calibration sphere or cube: Used for LiDAR intrinsic calibration, where the known geometry of the object helps characterize beam offset and divergence.
• Rate table and precision tilt platform: Used for high-accuracy IMU calibration to apply known angular rates and specific force vectors.

Calibration methodologies are categorized by when they occur relative to system operation.

• Offline Calibration: Performed in a controlled setup before deployment. It is precise but assumes parameters remain static. Examples include factory calibration or lab-based checkerboard routines.
• Online Calibration (Self-Calibration): Continuously estimates and updates parameters during normal operation. This is essential for handling parameter drift over time. Techniques often use SLAM (Simultaneous Localization and Mapping) or multi-sensor fusion algorithms (like a Kalman filter) to jointly estimate robot state and sensor parameters from perceived environmental features.

Sensor fusion is the computational process of combining data from multiple disparate sensors (e.g., camera, LiDAR, IMU) to produce a more accurate, complete, and reliable state estimate than any single sensor provides. Calibration is a prerequisite for effective fusion, as it ensures all sensor measurements are aligned to a common reference frame.

• Common Algorithms: Kalman filters (linear, extended, unscented), particle filters, and deep learning-based approaches.
• Key Benefit: Reduces uncertainty and compensates for individual sensor weaknesses (e.g., camera data in low light, IMU drift).

Camera intrinsics are the internal parameters of a camera model that define its geometric and optical properties. Camera extrinsics define the camera's pose (position and orientation) in a world coordinate system. Calibration solves for these parameters.

• Intrinsic Parameters: Focal length (fx, fy), principal point (cx, cy), and lens distortion coefficients (radial k1, k2, k3, tangential p1, p2).
• Extrinsic Parameters: A 3x3 rotation matrix (R) and a 3x1 translation vector (t).
• Calibration Target: Typically uses a known pattern (e.g., checkerboard, ChArUco board) to establish 3D-to-2D point correspondences.

System Identification (SysID) is the broader process of building mathematical models of dynamic systems from measured input-output data. Sensor calibration is a specific instance of SysID focused on the sensor's measurement model.

• Applies to: Actuator dynamics, friction models, and full robot dynamics, not just sensors.
• Methods: Includes frequency-domain analysis, step response tests, and optimization-based parameter fitting.
• Goal in Sim-to-Real: To create a high-fidelity simulation model whose parameters (mass, inertia, friction coefficients) match the real robot, reducing the reality gap.

In simulation and robotics, ground truth refers to the perfectly accurate, noise-free data about the state of the world known by the simulator or measured by a high-precision external system. It is the benchmark against which sensor measurements and calibration accuracy are evaluated.

• In Simulation: The simulator has direct access to ground truth object poses, robot states, and physics.
• In Real-World Calibration: Obtained via motion capture systems (e.g., Vicon), survey-grade laser trackers, or meticulously machined calibration fixtures.
• Primary Use: For training supervised perception models and quantitatively evaluating calibration and state estimation algorithms.

A Kalman filter is an optimal recursive algorithm that estimates the state of a linear dynamic system from a series of noisy measurements. It explicitly models sensor noise and system uncertainty, making accurate noise parameters from calibration critical for its performance.

• Two-Step Process: Prediction (projects state forward using a model) and Update (corrects prediction with new sensor data).
• Requires Calibrated Inputs: Sensor noise covariance matrices (R) and, for extended Kalman filters (EKF), accurate sensor models for the linearization step.
• Ubiquitous Applications: Visual-inertial odometry (VIO), GPS/INS navigation, and target tracking.

Hand-eye calibration is a specific calibration problem that solves for the fixed transformation between a robot's end-effector (the "hand") and a sensor (e.g., a camera, the "eye") mounted on it. This allows sensor data to be transformed into the robot's base coordinate frame for manipulation tasks.

• Classic Problem Formulation: Solves the equation AX = XB, where A is end-effector motion, B is observed sensor motion, and X is the unknown hand-eye transformation.
• Essential for: Vision-guided robotic picking, welding, and assembly where the camera moves with the robot arm.
• Methods: Include closed-form solutions (e.g., Tsai-Lenz), and iterative optimization approaches.