Epipolar Plane Image

An EPI is a dimensionality reduction of the 4D light field. It is constructed by fixing two of the four parameters (e.g., one spatial coordinate x and one angular coordinate v), resulting in a 2D image where the remaining two varying parameters (e.g., y and u) form the axes. This slice captures how a single scanline of the scene (y) appears from a continuum of viewpoints along one direction (u).

• Primary Function: Converts the complex 4D plenoptic function L(x, y, u, v) into an analyzable 2D image E(y, u) or E(x, v).
• Analogy: Similar to taking a slice through a 3D volume to get a 2D CT scan image.

The most defining characteristic of an EPI is that points in the 3D scene map to straight lines within the 2D EPI. The slope of each line is directly proportional to the depth of the corresponding scene point.

• Steep Slope: Indicates a point is near the camera (large disparity between views).
• Shallow Slope: Indicates a point is far from the camera (small disparity).
• Zero Slope (Vertical Line): Represents a point at infinite depth (no parallax).

This property transforms the problem of depth estimation or 3D reconstruction into the simpler task of detecting and analyzing line slopes in a 2D image.

EPIs are the computational foundation for many depth-from-light-field algorithms. By analyzing the slopes of lines in multiple EPIs (extracted from the same light field), a dense depth map can be reconstructed.

• Line Detection: Algorithms apply structure tensor analysis, PCA, or Hough transforms to detect line orientations in the EPI.
• Slope to Depth Conversion: The detected slope s is converted to depth Z using the known camera baseline B and focal length f: Z = B * f / s.
• Advantage: Provides dense, per-pixel depth estimates without explicit feature matching, as every scene point contributes to a line.

EPIs visually encode occlusion boundaries, which appear as line terminators or discontinuities in slope. Where one object occludes another in 3D space, the corresponding lines in the EPI will end, and new lines with different slopes will begin.

• Occlusion Cue: The point where a line ends provides a direct cue for depth discontinuities and object boundaries.
• Disocclusion: Regions hidden in some views but visible in others create complex patterns that advanced EPI analysis can resolve.
• Challenge: Occlusions complicate simple line-fitting, requiring robust algorithms that can handle broken or intersecting line structures.

The EPI is intrinsically linked to the epipolar geometry of a stereo or multi-view system. An EPI can be thought of as stacking the same epipolar line from a sequence of sub-aperture images (views) on top of each other.

• Epipolar Line: In a rectified stereo pair, a point in the left image has its corresponding point on the same horizontal scanline (epipolar line) in the right image.
• EPI as a Stack: An EPI for a fixed vertical coordinate y is created by taking the same horizontal scanline from every sub-aperture image (varying u) and stacking them vertically. The correspondence search along the 1D epipolar line in stereo becomes a line detection problem in the 2D EPI for light fields.

Beyond depth estimation, the linear structure of EPIs enables efficient view synthesis and digital refocusing.

• View Interpolation: To synthesize a novel view, one can extract appropriate slices from the EPI volume. Shifting along the angular axis (u) in the EPI directly corresponds to changing the viewpoint.
• Refocusing: Digital refocusing is achieved by shearing the EPI. Applying a horizontal shear to the E(y, u) image aligns lines of a chosen slope to be vertical, which corresponds to integrating light for a point at that specific depth, bringing it into focus. This is the core operation behind the Fourier slice photography theorem.
• Performance: Operations in EPI space are often more computationally efficient than full 4D light field manipulations.

A light field is a 4D or higher-dimensional vector function that describes the radiance of light rays flowing in every direction through every point in space. It is a practical, sampled representation of the full plenoptic function.

• Core Parameterization: Often represented using the two-plane parameterization, where a ray is defined by its intersections with two parallel planes (u,v for spatial coordinates and s,t for angular coordinates).
• Key Property: Encodes all visual information of a scene, enabling computational photography tasks like refocusing and view synthesis without explicit 3D models.
• Acquisition: Captured using plenoptic cameras (e.g., Lytro) or camera arrays.

Epipolar geometry describes the intrinsic projective relationship between two perspective views of a scene. It provides the geometric foundation for understanding correspondences in stereo imagery and light fields.

• Epipolar Plane: The plane formed by a 3D point in the scene and the two camera centers.
• Epipolar Line: The intersection of an epipolar plane with an image plane. Corresponding points in a stereo pair must lie on these conjugate lines.
• Fundamental Matrix: The algebraic representation (a 3x3 matrix) of this geometry, which maps a point in one image to its corresponding epipolar line in the other.

Ray space is a multi-dimensional coordinate system used to parameterize the light field, where each individual light ray is represented as a point. The Epipolar Plane Image is a 2D slice through this higher-dimensional space.

• Common Parameterization: The two-plane (u,v,s,t) parameterization is standard, where (u,v) is the spatial intersection and (s,t) is the angular direction.
• EPI as a Slice: By fixing one spatial coordinate (e.g., v) and one angular coordinate (e.g., t), you obtain a 2D (u,s) slice—this is the EPI.
• Visualization: In this slice, a scene point appears as a line whose slope is inversely proportional to its depth.

Disparity estimation is the process of calculating the horizontal shift (disparity) of corresponding points between two rectified stereo images. EPIs provide a direct, elegant framework for this computation.

• EPI Structure: In an EPI, a scene point manifests as a line. The slope of this line is directly related to inverse depth (1/Z).
• Algorithmic Advantage: Depth/disparity extraction from an EPI reduces to detecting these linear structures, often using line detection algorithms like the Hough Transform or analyzing oriented filters.
• Multi-View Consistency: EPIs enforce photo-consistency along these lines, providing a robust constraint for depth estimation across many input views.

View synthesis is the core computational task of generating photorealistic images of a scene from novel, unobserved camera viewpoints. EPIs are a classical tool for this in image-based rendering.

• EPI-Based Method: Given a set of densely sampled views on a line (a 1D camera array), the complete light slab can be reconstructed. Novel views are generated by extracting vertical slices from the reconstructed EPI volume.
• Handling Occlusions: The linear structures in EPIs explicitly reveal occlusion boundaries, where lines terminate, allowing algorithms to reason about visibility.
• Modern Context: While neural methods like Neural Radiance Fields (NeRF) now dominate, EPI theory underpins the multi-view consistency constraints these models must learn.

Plenoptic sampling refers to the theory governing the minimum sampling rates required to accurately capture and reconstruct a light field without aliasing, formalized by the plenoptic sampling theorem.

• Spatial-Angular Tradeoff: For a fixed sensor resolution, increasing angular resolution (more views) decreases spatial resolution (image detail). EPI analysis helps quantify this tradeoff.
• Bandwidth of Light Fields: The required sampling rate depends on the maximum parallax (depth variation) in the scene. Insufficient sampling leads to aliasing, visible as distorted lines in the EPI.
• Design Implication: This theorem guides the design of light field cameras and camera arrays, determining the necessary number of microlenses or cameras.