Camera Calibration

The intrinsic matrix encodes the camera's internal optical and geometric properties, independent of its position in space. It maps normalized camera coordinates to pixel coordinates.

• Focal Length (fx, fy): The distance from the optical center to the image plane, expressed in pixel units. Determines the scale of the projection.
• Principal Point (cx, cy): The pixel coordinates where the optical axis intersects the image sensor, typically near the image center.
• Skew Coefficient (s): Models the non-perpendicularity between the x and y pixel axes; usually zero for modern digital sensors.

The 3x3 matrix K is used to project 3D points from the camera's own coordinate frame onto the 2D sensor plane.

Real lenses deviate from the ideal pinhole model, introducing geometric distortions that must be modeled and corrected for accurate measurement.

• Radial Distortion (k1, k2, k3): Caused by the spherical shape of the lens, making straight lines appear curved. Barrel distortion bends lines outward; pincushion distortion bends them inward. Modeled as a polynomial function of the radial distance from the principal point.
• Tangential Distortion (p1, p2): Arises from misalignment between the lens and the image sensor plane during assembly. Causes the image to appear tilted or skewed.

Undistorting an image using these coefficients is a prerequisite for any metrology application where pixel coordinates must correspond to real-world measurements.

Extrinsic parameters define the rigid-body transformation that maps points from a fixed world coordinate frame to the camera's local coordinate frame. They establish the camera's pose in 3D space.

• Rotation Matrix (R): A 3x3 orthonormal matrix encoding the camera's orientation relative to the world axes. Often parameterized as a 3-element Rodrigues vector for compactness.
• Translation Vector (t): A 3x1 vector representing the displacement from the world origin to the camera's optical center.

The combined 3x4 matrix [R|t] is essential for stereo vision and hand-eye calibration, where the spatial relationship between the camera and a robot end-effector must be precisely known.

Reprojection error is the primary cost function minimized during calibration and the key metric for evaluating calibration quality. It quantifies the discrepancy between observed and predicted image points.

• Calculation: After estimating parameters, 3D world points are projected back onto the image plane using the solved K, R, t, and distortion coefficients. The Euclidean distance between this reprojected point and the originally detected feature point is the error.
• Root Mean Square (RMS) Reprojection Error: The square root of the mean of squared errors across all calibration images and points. A value below 0.5 pixels is generally considered good; values below 0.1 pixels indicate excellent calibration.
• High reprojection error signals poor feature detection, insufficient calibration images, or an inadequate distortion model.

The accuracy of estimated parameters depends heavily on the known geometry of the calibration target used to provide 2D-3D point correspondences.

• Checkerboard Pattern: The most common target. Internal corners are detected with sub-pixel accuracy. The known square size provides the metric scale linking pixels to world units (e.g., millimeters).
• Circle Grid: A pattern of circles on a contrasting background. Ellipse center detection is used instead of corner detection, which can be more robust to defocus blur.
• ChArUco Boards: A hybrid combining a checkerboard with ArUco markers. Allows calibration even when the full board is not visible, as each marker encodes a unique ID for unambiguous point correspondence.
• The target must be manufactured with high precision, as any deviation in its physical dimensions directly propagates to errors in the calibrated parameters.

The de facto standard algorithm for camera calibration, proposed by Zhengyou Zhang in 1999, requires observing a planar pattern from at least two different orientations. It does not require knowledge of the pattern's motion.

• Closed-Form Solution: First, a linear solution for the intrinsic matrix is computed by exploiting the homography between the model plane and its image.
• Non-Linear Refinement: The closed-form solution is used as an initial guess for Levenberg-Marquardt optimization, which iteratively minimizes the total reprojection error over all parameters, including distortion coefficients.
• The method's elegance lies in its practicality: moving a flat target by hand provides sufficient constraints to solve for all intrinsic and extrinsic parameters without expensive 3D calibration rigs.

The internal characteristics of the camera model, represented by a 3x3 matrix. These parameters are fixed for a given lens and sensor combination.

• Focal length (fx, fy): The distance from the lens to the image sensor, measured in pixels.
• Principal point (cx, cy): The pixel coordinates where the optical axis intersects the image sensor.
• Skew coefficient: A measure of the angle between the x and y pixel axes, typically zero for modern sensors.
• Lens distortion coefficients: Radial (k1, k2, k3) and tangential (p1, p2) parameters that model barrel and pincushion distortion.

The external geometric transformation that maps points from a 3D world coordinate system to the 3D camera coordinate system. This is defined by a rigid-body transformation.

• Rotation Matrix (R): A 3x3 orthonormal matrix defining the camera's orientation relative to the world frame.
• Translation Vector (t): A 3x1 vector defining the camera's position in world coordinates.
• Together, [R|t] forms a 3x4 matrix that transforms homogeneous world points into camera-centric coordinates before projection.

The idealized mathematical model that forms the basis of most calibration algorithms. It assumes all light rays pass through a single infinitesimal aperture and project onto an image plane.

• Maps a 3D point (X, Y, Z) to a 2D pixel (u, v) via the equation: s * [u, v, 1]^T = K * [R|t] * [X, Y, Z, 1]^T.
• K is the intrinsic matrix, [R|t] is the extrinsic matrix, and s is a projective scale factor.
• This linear model is applied after non-linear lens distortion has been corrected.

A widely adopted planar calibration technique that estimates intrinsic parameters by observing a flat pattern, typically a checkerboard, from multiple unknown orientations.

• Requires at least 3 images of a planar pattern held at different angles and distances.
• Solves for the homography between the model plane and its image to compute a closed-form initial solution.
• Refines all parameters using Levenberg-Marquardt non-linear optimization to minimize reprojection error.
• Does not require expensive 3D calibration rigs, making it practical for factory-floor deployment.

The primary quantitative metric for assessing calibration accuracy. It measures the Euclidean distance in pixels between an observed feature point and its projected position based on the estimated model.

• Root Mean Square (RMS) reprojection error is the standard summary statistic; a value under 0.5 pixels is generally considered excellent.
• Minimizing this geometric error is the objective function of non-linear optimization during calibration.
• High reprojection error indicates poor corner detection, motion blur, or an insufficient number of calibration images.

A specialized calibration process that determines the rigid transformation between a robot's end-effector and a mounted camera, or between a robot base and a fixed camera.

• Solves the AX = XB equation, where A is robot motion, B is camera motion, and X is the unknown hand-eye transform.
• Essential for robotic guidance and pick-and-place applications where objects detected in the image must be grasped by the robot.
• Two common configurations: eye-in-hand (camera moves with the gripper) and eye-to-hand (camera is stationary, observing the workspace).