Epipolar Geometry

The core principle of epipolar geometry states that for a given point x in the first image, its corresponding point x' in the second image must lie on a specific line called the epipolar line. This constraint drastically reduces the search space for feature matching from the entire second image to a single line. It is algebraically expressed by the fundamental matrix F or the essential matrix E via the equation x'ᵀ F x = 0 for uncalibrated cameras.

Key geometric entities define the epipolar framework:

• Epipole (e, e'): The point of intersection of the baseline (line connecting the two camera centers) with each image plane. It is the projection of one camera center onto the other's image.
• Epipolar Line (l, l'): The line in an image plane along which the corresponding point for a feature in the other image must lie. All epipolar lines in one image intersect at that image's epipole, forming a pencil of lines.

A 3x3 matrix of rank 2 that encapsulates the epipolar geometry for a pair of uncalibrated cameras. It maps a point in one image to its corresponding epipolar line in the other. Properties include:

• Algebraic Constraint: For corresponding points x and x', the equation x'ᵀ F x = 0 holds.
• Epipolar Lines: The epipolar line in the second image is l' = F x. In the first image, l = Fᵀ x'.
• Epipoles: The epipoles satisfy F e = 0 and Fᵀ e' = 0. It has 7 degrees of freedom and can be estimated from point correspondences using algorithms like the 8-point algorithm followed by enforcement of the rank-2 constraint.

A 3x3 matrix that describes the epipolar geometry between two calibrated cameras. It encodes the relative rotation R and translation t (up to scale) between the cameras. It is related to the fundamental matrix by E = K'ᵀ F K, where K and K' are the camera intrinsic matrices. Key properties:

• It is derived from the normalized image coordinates of corresponding points.
• It satisfies the epipolar constraint: x̂'ᵀ E x̂ = 0, where x̂ are normalized coordinates.
• It can be decomposed via Singular Value Decomposition (SVD) to recover the four possible solutions for R and t, with the correct one identified by the cheirality constraint (ensuring points are in front of both cameras).

The plane formed by a 3D world point X and the two camera centers C and C'. This plane intersects each image plane along the corresponding epipolar lines. As the 3D point X moves along a ray from the first camera center, its projection in the first image remains fixed, but its projection in the second image moves along the fixed epipolar line. This illustrates the depth ambiguity along a single ray, which is resolved through triangulation with a second view.

Epipolar geometry is not just theoretical; it is critical for robust, efficient algorithms:

• Stereo Correspondence & Depth Maps: Restricting the search for matches to epipolar lines (rectified images make these lines horizontal) enables efficient stereo matching and dense disparity map computation.
• Structure from Motion (SfM) & Visual Odometry (VO): It is used to estimate the fundamental or essential matrix from feature matches, which is then decomposed to get initial camera motion for bundle adjustment.
• Outlier Rejection with RANSAC: The epipolar constraint provides a geometric model (F or E) that RANSAC uses to robustly identify and reject incorrect feature matches (outliers).

Epipolar geometry reduces the correspondence search from a 2D area to a 1D line. Given a point in one image, its match in the second image must lie on the corresponding epipolar line. This constraint is the basis for:

• Stereo matching algorithms that compute dense disparity maps.
• Depth estimation for autonomous vehicles and robotics.
• Efficient computation in multi-view stereo (MVS) pipelines.

The essential matrix and fundamental matrix, derived from epipolar constraints, are used to recover the relative camera pose (rotation and translation) between two views. This is a critical step in:

• Structure from Motion (SfM) to initialize camera positions.
• Visual Odometry (VO) for tracking a moving camera.
• Simultaneous Localization and Mapping (SLAM) systems. The eight-point algorithm is a classic linear method for solving for these matrices from point correspondences.

Once camera poses are known via epipolar geometry, the 3D position of a scene point can be determined through triangulation. The point is found at the intersection of the two rays back-projected from its corresponding image points. This is fundamental to:

• Creating sparse 3D point clouds in SfM.
• Dense reconstruction for digital twins and photogrammetry.
• Bundle adjustment, which jointly refines 3D points and camera poses by minimizing reprojection error.

Feature matching produces many incorrect correspondences (outliers). Epipolar geometry provides a model (the fundamental matrix) for robust estimation. RANSAC (Random Sample Consensus) uses this model to:

• Randomly sample minimal point sets (e.g., 8 points) to compute a candidate matrix.
• Classify all other matches as inliers or outliers based on their distance to the epipolar line.
• Iteratively select the model with the most inliers, effectively filtering out mismatches.

Image rectification warps a stereo pair so that corresponding epipolar lines become horizontal and aligned. This transforms the correspondence search into a simple horizontal scan along image rows, which:

• Dramatically simplifies and accelerates stereo matching.
• Enables efficient hardware implementation for real-time depth sensing.
• Is a standard pre-processing step for many depth from stereo algorithms used in robotics and 3D sensing.

Epipolar constraints help distinguish static scene structure from independent moving objects. Points on a rigidly moving object will satisfy a different fundamental matrix than the background. This principle extends to:

• Multi-view geometry with more than two cameras, governed by the trifocal tensor.
• Detecting egomotion versus independent object motion for automotive safety.
• Dynamic scene reconstruction, where multiple motions must be disentangled.

The Fundamental Matrix (F) is a 3x3 matrix of rank 2 that encapsulates the epipolar geometry between two uncalibrated cameras. It maps a point in one image to its corresponding epipolar line in the other image, satisfying the equation x'ᵀ F x = 0 for corresponding points x and x'.

• Key Property: Requires only pixel coordinates; no camera calibration needed.
• Estimation: Typically computed from at least 7 or 8 point correspondences using algorithms like the 8-point algorithm or RANSAC to handle outliers.
• Application: The foundation for uncalibrated stereo vision, image rectification, and structure-from-motion pipelines.

The Essential Matrix (E) is a 3x3 matrix that describes the epipolar geometry between two calibrated cameras. It encodes the relative rotation (R) and translation (t) (up to scale) between the cameras, derived from the equation E = [t]ₓ R, where [t]ₓ is the skew-symmetric matrix of the translation vector.

• Key Property: Derived from the Fundamental Matrix when camera intrinsics (K, K') are known: E = K'ᵀ F K.
• Decomposition: Can be decomposed via Singular Value Decomposition (SVD) to extract the four possible solutions for R and t, with the correct one determined by enforcing positive depth (cheirality constraint).
• Application: Critical for recovering the precise metric motion between calibrated cameras in Visual Odometry and SLAM.

An Epipole is the point of intersection of the line joining the two camera centers (the baseline) with an image plane. In a stereo pair, the epipole in the second image is the projection of the first camera's center, and vice-versa.

• Geometric Meaning: All epipolar lines in an image converge at its epipole.
• Special Cases: If the cameras are in a canonical configuration (translation only along the x-axis), the epipoles are at infinity, and the epipolar lines become parallel (simplifying stereo matching).
• Practical Insight: The location of the epipole indicates the direction of the other camera relative to the image plane.

An Epipolar Line is the projection of the ray from one camera center through a 3D point onto the other image plane. For a given point x in the first image, its corresponding point x' in the second image is constrained to lie on the epipolar line l' = Fx.

• Constraint: This is the core epipolar constraint that reduces the search for correspondences from a 2D region to a 1D line.
• Image Rectification: A process that warps image pairs so that corresponding epipolar lines become horizontal and aligned, turning the correspondence search into a simple horizontal scanline search.
• Use in Stereo Matching: Drastically reduces computational cost and ambiguity in dense disparity map estimation.

Triangulation is the process of computing the 3D world coordinates of a point given its 2D projections in two or more images and the known camera poses (or essential matrix). It finds the intersection of the back-projected rays from each camera center through the image points.

• Challenge: Due to noise, the rays often do not intersect perfectly. The solution is found by minimizing the reprojection error.
• Linear Methods: Include the Direct Linear Transform (DLT) for triangulation.
• Optimal Methods: Use non-linear optimization (e.g., via Gauss-Newton) to find the 3D point that minimizes the sum of squared reprojection errors, providing the Maximum Likelihood Estimate.
• Application: The fundamental step for building 3D models in Structure from Motion and Multi-View Stereo.

The Five-Point Algorithm is a minimal solution for estimating the Essential Matrix from five point correspondences between two calibrated images. It is more efficient and stable than the eight-point algorithm for the essential matrix.

• Minimal Solution: Five correspondences are the theoretical minimum needed to solve for the five degrees of freedom in the essential matrix (3 for rotation, 2 for translation direction; scale is ambiguous).
• Process: The algorithm formulates a system of polynomial equations from the constraints, which is solved to yield up to 10 possible solutions for E. The correct one is selected using the cheirality constraint.
• Advantage: Highly robust in real-world applications like Visual Odometry, where efficient, robust motion estimation from few features is critical.