Plenoptic Sampling Theorem

The theorem specifies minimum sampling rates in two distinct domains to prevent aliasing:

• Spatial Sampling (Δx, Δy): The density of sample points on the camera sensor plane. This must be fine enough to capture the highest spatial frequencies present in the scene.
• Angular Sampling (Δu, Δv): The density of sample points across the camera aperture (or across different camera positions). This must be fine enough to capture the highest angular frequency variations in the light field. A failure to meet either requirement results in irrecoverable information loss and visual artifacts like jagged edges (spatial aliasing) or incorrect parallax (angular aliasing) in synthesized views.

The theorem's derivations rely on modeling the plenoptic function as bandlimited. This is a critical, simplifying assumption that:

• Treats the real-world light field as having a maximum frequency in both spatial and angular dimensions.
• Enables the application of classic Nyquist-Shannon sampling theory to the 4D signal. In practice, real scenes contain occlusions and sharp edges that introduce high frequencies, making them not strictly bandlimited. This theoretical idealization guides system design but requires practical oversampling and regularization techniques to handle real-world discontinuities.

A core implication is the fixed bandwidth product for a given sensor. For a camera with N total pixels, the theorem defines a trade-off: N ≥ (2B_x * 2B_u) * (2B_y * 2B_v) Where B are the spatial and angular bandwidths. This creates the spatial-angular resolution trade-off:

• A high angular resolution (many distinct viewpoints, good for refocusing) forces a low spatial resolution (blurry sub-aperture images).
• A high spatial resolution (sharp images) forces a low angular resolution (limited viewpoint/refocusing capability). This trade-off fundamentally limits the capabilities of single-snapshot plenoptic cameras and informs the design of multi-camera arrays.

Given sufficiently sampled rays, the theorem provides the framework for perfect reconstruction of the continuous light field. This is achieved through a 4D convolution of the discrete samples with an ideal sinc interpolation filter. The reconstruction formula in ray space L(x, u) is: L(x, u) = Σ Σ L_s[n, m] * sinc(x/Δx - n) * sinc(u/Δu - m) Where L_s are the sampled rays. In practice, ideal sinc filters are infeasible, so approximations like 4D linear or bilinear interpolation are used for tasks like novel view synthesis, leading to a trade-off between computational cost and reconstruction quality.

The theorem directly informs the engineering of capture systems:

• Plenoptic Camera Design: Dictates the pitch and arrangement of the microlens array relative to the main sensor to satisfy angular sampling.
• Camera Array Design: Determines the minimum baseline between cameras and the required number of cameras to achieve a desired angular sampling for a given scene depth range.
• Coded Apertures & Masks: Explores non-uniform sampling patterns that can relax the strict requirements of regular sampling by exploiting sparsity or prior knowledge about the light field. It establishes that there is no 'free lunch'—high-quality light field processing requires a proportional amount of raw captured data.

The sampling requirements can be expressed in terms of observable scene properties, making them practical for engineers. The necessary angular sampling rate Δu is related to:

• Scene Depth Range: Δu ≤ (Δx * f) / (2 * D_max * (1/f - 1/D_min)) where f is focal length, and D is depth. Deeper scenes require finer angular sampling.
• Maximum Disparity: The theorem shows that to avoid aliasing, the angular sampling must be finer than the maximum disparity (in pixels) between extreme views. This directly links light field theory to multiview stereo and epipolar geometry, where disparity provides the depth signal that must be adequately sampled.

A light field is a vector function that describes the amount of light flowing in every direction through every point in space. It is a practical 4D or 5D representation (position + direction) of the full plenoptic function, forming the core dataset to which the sampling theorem is applied.

• Parameterization: Commonly represented using the two-plane parameterization, where a ray is defined by its intersections with two parallel planes.
• Acquisition: Captured by devices like plenoptic cameras or arrays of conventional cameras.
• Application: Enables post-capture effects like refocusing and viewpoint shifting.

The spatial-angular tradeoff is a fundamental engineering constraint in light field acquisition. For a fixed sensor resolution, increasing the density of angular samples (more ray directions) necessarily reduces the density of spatial samples (image resolution).

• Theorem Implication: The Plenoptic Sampling Theorem quantifies this tradeoff, defining the minimum sampling rates in both domains to avoid aliasing.
• System Design: Dictates the design of microlens arrays in plenoptic cameras, balancing the number of microlenses (spatial resolution) against pixels per microlens (angular resolution).

Ray space is a multi-dimensional coordinate system used to parameterize all light rays. The Lumigraph is a specific, structured 4D data structure built within ray space for efficient light field rendering and interpolation.

• Representation: Organizes captured rays based on their intersection with a simple bounding geometry (e.g., a cube).
• Connection to Sampling: The organization of rays in a Lumigraph directly relates to the structured sampling grids analyzed by the Plenoptic Sampling Theorem. Proper sampling ensures the Lumigraph can be queried to generate novel views without artifacts.

View synthesis is the computational process of generating photorealistic images from unseen camera viewpoints. Novel view generation is the core task, relying on densely sampled input data.

• Goal: The primary application enabled by adequately sampled light fields.
• Theorem's Role: Defines the minimum input requirements (number of images, camera positions) to synthesize a continuous range of novel views without aliasing or ghosting artifacts. Insufficient sampling leads to broken parallax and incorrect occlusion handling.

An Epipolar Plane Image (EPI) is a 2D slice through a 4D light field where one spatial and one angular dimension are fixed. It visualizes rays within a single epipolar plane.

• Structure: Points in the scene manifest as straight lines in an EPI. The slope of the line is inversely proportional to the depth of the corresponding scene point.
• Sampling Analysis: EPI analysis is a direct tool for understanding sampling requirements. The Plenoptic Sampling Theorem ensures EPIs are sampled finely enough to correctly reconstruct these linear structures, which is equivalent to correctly estimating disparity and depth.

A plenoptic camera (or light field camera) is a physical implementation device for sampling the light field. Its key component is a microlens array placed between the main lens and the sensor.

• Function: Each microlens samples the pupil of the main lens, capturing a small sub-aperture image of the scene from a slightly different viewpoint. The array of microlenses provides the spatial sampling.
• Direct Link to Theorem: The design specifications of the microlens array—its pitch (spacing) and the number of pixels behind each lens—are direct consequences of applying the Plenoptic Sampling Theorem to the camera's desired field of view and depth range.