Epipolar Geometry

The epipolar constraint is the fundamental rule of epipolar geometry. It states that for a given point x in the first image, its corresponding point x' in the second image must lie on a specific line called the epipolar line. This reduces the search for correspondences from the entire second image to a single line, dramatically simplifying stereo matching and structure-from-motion algorithms.

• Mathematical Basis: The constraint is encoded in the fundamental matrix F, where x'ᵀ F x = 0 for all corresponding points.
• Practical Impact: This is why stereo cameras are often rectified—to make epipolar lines horizontal and parallel, enabling efficient 1D search.

Epipoles and epipolar lines are the primary geometric entities. The epipole (e, e') is the point in one image where the projection of the other camera's center is located. All epipolar lines in an image radiate from its epipole.

• Epipolar Line: For a point x in the left image, the epipolar line l' in the right image is the projection of the ray through x and the left camera center. The corresponding point x' must lie on l'.
• Visualization: In a rectified stereo pair, the epipoles are at infinity, and all epipolar lines become horizontal and parallel.

The fundamental matrix F is the 3x3 algebraic representation of epipolar geometry for uncalibrated cameras. It encapsulates the projective relationship between two views.

• Properties: Rank 2, with 7 degrees of freedom. It can be estimated from as few as 7 point correspondences (8-point algorithm is standard).
• Function: Given a point x in image 1, the corresponding epipolar line in image 2 is l' = Fx. The constraint is x'ᵀ F x = 0.
• Use Case: Essential for uncalibrated stereo matching, image rectification, and projective reconstruction.

The essential matrix E is the specialization of the fundamental matrix for calibrated cameras (where intrinsic parameters are known). It relates corresponding points in normalized image coordinates.

• Relation to F: E = K'ᵀ F K, where K and K' are the camera calibration matrices.
• Properties: Rank 2, with two equal non-zero singular values (SVD constraint). It has 5 degrees of freedom (3 for rotation, 2 for translation direction).
• Critical Function: From E, one can directly extract the relative rotation (R) and translation (t) (up to scale) between the two cameras, which is the core of visual odometry and SLAM.

Stereo rectification is the process of applying 2D projective transforms (homographies) to a pair of images so that their epipolar lines become horizontal, collinear, and aligned. This transforms a general stereo configuration into a simple, canonical one.

• Purpose: Converts the correspondence search from a 2D problem to a 1D problem along horizontal scanlines.
• Algorithm: Computes rectifying homographies from the fundamental or essential matrix to make the epipoles go to infinity.
• Result: Produces the standard rectified image pair where disparity (horizontal shift) is directly proportional to inverse depth: Z = f * B / d, where f is focal length, B is baseline, and d is disparity.

Epipolar geometry is not theoretical; it is the engine behind critical 3D computer vision systems.

• Dense Stereo Matching: Provides the 1D search constraint for algorithms like Semi-Global Matching (SGM).
• Structure from Motion (SfM): The fundamental matrix is used to initialize two-view geometry and triangulate initial 3D points.
• Visual SLAM & Odometry: The essential matrix provides the relative pose between frames for camera tracking.
• Multi-View Stereo (MVS): Enforces photo-consistency checks along epipolar lines to build dense 3D models.
• Novel View Synthesis: Used to warp pixels from source views to a target viewpoint, as in image-based rendering.

The epipole is the point of intersection between the image plane and the line connecting the two camera centers (the baseline). In a stereo pair, the epipole in the right image is the projection of the left camera center, and vice-versa. It is the vanishing point for all epipolar lines in that image.

• Properties: All epipolar lines in an image radiate from its epipole.
• Special Case: For parallel, rectified cameras, the epipoles are at infinity, making epipolar lines horizontal and parallel.

An epipolar line is the projection of the epipolar plane onto an image. For a given point x in the left image, its corresponding point x' in the right image is constrained to lie along a specific epipolar line. This reduces the correspondence search from a 2D image area to a 1D line.

• Constraint: The fundamental matrix F encapsulates this relationship: x'ᵀ F x = 0.
• Application: This is the core of efficient stereo matching and depth estimation algorithms.

The fundamental matrix is a 3x3 rank-2 matrix that encapsulates the projective geometric relationship between two uncalibrated views. It maps a point in one image to its corresponding epipolar line in the other image.

• Equation: For corresponding points x and x', x'ᵀ F x = 0.
• Properties: Contains only intrinsic projective geometry. Can be estimated from point correspondences using algorithms like the 8-point algorithm or RANSAC.

The essential matrix is a 3x3 matrix that describes the geometric relationship between two calibrated cameras. It is a specialization of the fundamental matrix when intrinsic camera parameters (like focal length) are known.

• Relation: E = K'ᵀ F K, where K and K' are the camera calibration matrices.
• Properties: Can be decomposed directly into the relative rotation R and translation t (up to scale) between the two camera poses.

Stereo rectification is the process of applying a 2D projective transformation (a homography) to a pair of stereo images so that corresponding epipolar lines become collinear and horizontal. This simplifies the correspondence search to a horizontal scanline match.

• Purpose: Transforms a general stereo configuration into a canonical, parallel one.
• Result: Epipoles are moved to infinity, enabling efficient dense disparity computation.

Triangulation is the process of determining the 3D world coordinates of a point given its 2D projections in two or more images and the known camera poses. It is the inverse of projection.

• Epipolar Foundation: Requires solving for the intersection of the two back-projected rays from corresponding image points. In the presence of noise, these rays may not intersect, requiring a least-squares solution.
• Direct Link: Epipolar geometry ensures the rays are coplanar (in the epipolar plane), making triangulation possible.