Intrinsic Calibration

The foundational geometric model mapping 3D world points to a 2D image plane through a single point—the optical center. Governed by the intrinsic matrix K, it defines:

• Focal length (fx, fy): Distance from optical center to image plane, expressed in pixel units. Determines magnification.
• Principal point (cx, cy): The intersection of the optical axis with the image sensor, ideally at the image center but often offset due to manufacturing tolerances.
• Skew coefficient (s): Accounts for non-perpendicular pixel axes, typically zero in modern digital sensors.

A 3D point [X, Y, Z] projects to pixel coordinates [u, v] via: u = fx*(X/Z) + cx and v = fy*(Y/Z) + cy. This linear projection is the starting point before modeling non-linear distortions.

A non-linear optical aberration causing straight lines in the world to appear curved in the image, most pronounced toward the edges. Caused by the spherical shape of lens elements. Modeled using a polynomial function of the radial distance r from the principal point:

• Barrel distortion: Image magnification decreases with distance from the optical axis. Common in wide-angle and fisheye lenses.
• Pincushion distortion: Magnification increases with distance. Typical in telephoto lenses.
• Mustache distortion: A complex combination of barrel and pincushion.

Corrected using coefficients k1, k2, k3 in the Brown-Conrady model: x_corrected = x(1 + k1*r² + k2*r⁴ + k3*r⁶). For severe fisheye distortion, higher-order polynomials or specialized models like the Kannala-Brandt model are required.

A non-linear aberration arising from physical misalignment between the lens assembly and the image sensor plane during manufacturing. Unlike radial distortion, it is not rotationally symmetric.

• Cause: Lens elements not perfectly parallel to the sensor, or decentered lens components.
• Effect: Points appear displaced in a direction tangential to concentric circles around the principal point, creating a wedge-shaped distortion pattern.
• Correction: Modeled with two coefficients p1, p2 in the Brown-Conrady model: x_corrected = x + [2*p1*x*y + p2*(r² + 2x²)].

While typically smaller in magnitude than radial distortion, tangential distortion must be calibrated for high-accuracy metrology and stereo vision applications where sub-pixel precision is required.

The primary quantitative metric for assessing intrinsic calibration quality. Defined as the Euclidean distance in pixels between an observed feature point in the image and the projected location of the corresponding 3D world point using the estimated camera parameters.

Calculation: For each detected corner p_observed and its known 3D counterpart P_world, compute p_projected = project(K, dist_coeffs, P_world), then error = ||p_observed - p_projected||.

Interpretation:

• < 0.1 pixels: Excellent calibration, suitable for precision metrology.
• 0.1 - 0.5 pixels: Good, acceptable for most computer vision tasks.
• > 1.0 pixel: Indicates poor calibration, blurry images, or incorrect pattern detection.

Root Mean Square (RMS) reprojection error across all images is the standard summary statistic reported by calibration toolboxes.

Techniques for estimating intrinsic parameters without a known calibration target, relying solely on correspondences between images of a static scene or on properties of the scene itself.

Key approaches:

• Kruppa equations: Derive constraints on intrinsics from the epipolar geometry of image pairs, specifically from the relationship between the fundamental matrix and the absolute conic.
• Mendonça-Cipolla: A stratified approach recovering the absolute quadric from projective reconstruction.
• Plane at infinity methods: Use vanishing points of parallel lines in architectural scenes to estimate focal length and principal point.
• Deep learning-based: CNNs trained on large datasets to directly regress intrinsic parameters from single images, useful for in-the-wild internet photos.

Limitations: Generally less accurate than target-based methods. Sensitive to noise and degenerate motions. Used when physical calibration targets are impractical, such as in legacy video footage or consumer photography.

The process of determining the rigid-body transformation—a 6-DOF pose comprising a 3x3 rotation matrix and a 3x1 translation vector—that defines the spatial relationship between the coordinate frames of two or more distinct sensors. While intrinsic calibration corrects errors within a single sensor, extrinsic calibration aligns multiple sensors to a common reference frame.

• Target-based methods use known calibration patterns (checkerboards, ArUco markers) visible to both sensors.
• Target-less methods estimate the transform by aligning natural features in overlapping fields of view, often using motion-based hand-eye calibration.
• The combined transform enables projecting a LiDAR point onto the corresponding camera pixel for sensor fusion.

The mathematical abstraction that maps 3D world points to 2D image coordinates. The standard pinhole camera model is defined by the intrinsic matrix K, which contains the focal lengths (f_x, f_y) and principal point (c_x, c_y). This linear projection is then distorted by a nonlinear lens distortion model.

• Radial distortion causes straight lines to appear curved (barrel or pincushion), modeled by coefficients k1, k2, k3.
• Tangential distortion arises from lens-sensor misalignment, modeled by coefficients p1, p2.
• Full projection: x_distorted = K * (R * X_world + t), followed by distortion application.

A probabilistic graphical model framework used to solve large-scale state estimation problems, including simultaneous intrinsic and extrinsic calibration. The problem is represented as a bipartite graph where variable nodes (sensor parameters, poses) are connected by factor nodes (probabilistic constraints from measurements).

• The optimal parameter set is found by solving a nonlinear least squares problem, typically via Gauss-Newton or Levenberg-Marquardt.
• Enables self-calibration during operation by treating intrinsic parameters as optimizable variables alongside robot poses and landmark positions.
• Libraries like GTSAM and g2o provide efficient sparse solvers for these problems.

A mathematical assessment of whether the intrinsic parameters of a sensor can be uniquely and unambiguously determined from the available measurement data. A parameter is observable if changes in its value produce distinguishable changes in the sensor output.

• Degenerate motions (e.g., pure rotation without translation for a monocular camera) render certain parameters like focal length and metric scale unobservable.
• Observability rank condition tests whether the Jacobian of the measurement function with respect to the parameters has full column rank.
• Critical for designing calibration trajectories that sufficiently excite all parameter dimensions.

The mathematical process of determining how the quantified uncertainty in estimated intrinsic parameters (e.g., the covariance of focal length) propagates through downstream perception pipelines to affect the final measurement uncertainty.

• A calibrated camera's reprojection error covariance can be derived from the intrinsic parameter covariance using first-order Taylor expansion: Σ_pixel = J * Σ_intrinsics * J^T.
• This enables error budgeting—allocating acceptable uncertainty to calibration versus other error sources.
• Essential for safety-critical applications where a 3D position estimate must be reported with a guaranteed confidence bound.

An advanced technique that performs online intrinsic calibration concurrently with ego-motion estimation and mapping, eliminating the need for a dedicated offline calibration procedure. The sensor's internal parameters are treated as time-varying or static state variables within a SLAM framework.

• Particularly valuable for long-term autonomy where thermal cycling and vibration cause parameter drift.
• A factor graph formulation adds prior factors on intrinsics and reprojection error factors that constrain both the pose and the calibration parameters.
• Enables plug-and-play sensor integration without manual calibration targets.