Focal Stack

The primary application is synthesizing a single all-in-focus image from a focal stack. Algorithms analyze each image to select the sharpest pixels at each spatial location, combining them into a final composite with a depth of field extending from the nearest to the farthest captured plane. This is critical in macro photography and microscopy, where lens physics severely limits the depth of field in a single shot. The process involves:

• Multi-scale pyramid analysis to assess local sharpness (e.g., using Laplacian variance).
• Weight map generation to blend pixels seamlessly.
• Advanced fusion to avoid artifacts like haloing at depth discontinuities.

Focal stacks provide the core data for Depth-from-Defocus algorithms, which estimate a dense depth map by modeling how blur (defocus) changes with focus distance. Unlike stereo matching, DFD uses a single camera and does not require textured surfaces, making it effective for smooth, untextured objects. The process involves:

• Modeling the Circle of Confusion (CoC) as a function of depth and lens parameters.
• Fitting the observed blur kernel at each pixel across the stack.
• Solving an inverse problem to recover the per-pixel depth. This technique is used in industrial inspection and some smartphone portrait modes for depth estimation.

By simulating the integration of rays from the captured focal stack, one can synthetically re-focus an image after capture. This is a simplified form of light field rendering. The algorithm effectively shifts and blends slices of the stack to bring any depth plane into sharp focus while blurring others, emulating a shallow depth of field effect. Key implementations include:

• Shift-and-add algorithms for fast, approximate refocusing.
• More sophisticated integration using estimated depth maps for higher quality.
• Applications in computational microscopy and consumer photography apps.

In scientific imaging, particularly transmission electron microscopy (TEM) and confocal microscopy, a focal stack (termed a 'through-focus series') is used to reconstruct high-resolution 3D structure. The changing blur contains phase information. Algorithms like:

• Tomographic reconstruction from a tilt series, where focus change provides complementary data.
• Wavefront reconstruction techniques (e.g., ptychography) that use defocus diversity to solve for the complex optical field of a specimen. This enables nanoscale 3D imaging of materials and biological samples.

A focal stack can be used to deblur a single, defocused image by leveraging the information present in other slices of the stack. By identifying the in-focus regions across the stack, the algorithm can construct a point spread function (PSF) model for out-of-focus areas and perform non-blind deconvolution. This is valuable for:

• Astronomical imaging, where atmospheric turbulence and optical imperfections cause blur.
• Restoring historical photographs with focus errors.
• Improving the effective resolution of microscope images beyond the diffraction limit in some modalities.

The way defocus blur manifests can reveal surface properties. Specular highlights and translucent materials blur differently than diffuse surfaces. By analyzing the blur kernel morphology across a focal stack, algorithms can infer:

• Surface roughness and glossiness (material BRDF properties).
• Subsurface scattering parameters in translucent objects like marble or skin.
• Fine geometric detail that is below the spatial resolution of the sensor but affects the blur profile. This finds use in automated quality inspection and digital asset creation for accurate material capture.

A light field is a 4D or higher-dimensional vector function that describes the amount of light flowing in every direction through every point in space. It is a sampled representation of the full plenoptic function.

• Core Data for Focal Stacks: A focal stack can be synthetically generated from a captured light field by integrating rays that converge at different virtual focal planes.
• Acquisition: Requires specialized hardware like a plenoptic camera with a microlens array.
• Applications: Enables post-capture refocusing, viewpoint shifting, and depth estimation without multiple physical exposures.

The plenoptic function is the complete theoretical description of all visual information in a scene. It is a 7D function: I = f(x, y, z, θ, φ, λ, t), representing light intensity at every 3D position (x,y,z), for every direction (θ,φ), for every wavelength λ, and at every time t.

• Theoretical Foundation: A focal stack and a light field are both finite, lower-dimensional samples of this infinite function.
• Goal of Modeling: The aim of plenoptic function modeling is to reconstruct or approximate this complete function from sparse samples to enable photorealistic view synthesis.

Depth from Defocus is a passive depth estimation technique that calculates scene geometry by analyzing the differences in blur (defocus) between two or more images captured with different focus settings.

• Primary Use of Focal Stacks: A focal stack is the ideal input for DfD algorithms. By measuring how a point's blur circle changes across the stack, its depth can be precisely triangulated.
• Contrast to Stereo: Unlike stereo vision, which uses parallax from viewpoint shift, DfD uses information from a single viewpoint with varying optical parameters.
• Algorithm Steps: Typically involves modeling the point spread function (PSF) of the camera and solving an inverse problem to find the depth map that best explains the observed blur variation.

Extended Depth of Field is a computational imaging technique that combines a focal stack to produce a single, fully focused image where all objects from near to far appear sharp.

• Direct Application: This is one of the most common practical uses for a captured focal stack.
• Fusion Algorithms: Techniques like wavelet-based fusion or Laplacian pyramid blending select the sharpest pixels from each image in the stack based on local focus metrics (e.g., gradient magnitude).
• Microscopy & Macrophotography: Critical in fields like microscopy and macro photography, where the physical depth of field is extremely shallow.

Digital refocusing is the process of synthetically changing the focal plane of a photograph after it has been captured, simulating the effect of having focused the camera lens at a different distance.

• Enabled by Light Fields: A primary feature of light field photography. From a single light field capture, a full focal stack can be rendered computationally.
• Mechanism: Achieved by integrating or shifting the sub-aperture images extracted from the light field to simulate the convergence of light rays at a new plane.
• User Interaction: Allows for interactive 'focus picking' in post-production, useful for portrait photography and creative control.

A focal sweep is a specific capture technique where the camera's focus distance is continuously varied during a single sensor exposure, effectively integrating a focal stack into one blurred image.

• Motion Blur Analogy: Similar to motion blur, but applied to the focus parameter instead of camera position.
• Computational Decoding: Specialized deconvolution algorithms can recover a depth map or even a full focal stack from this single, purposefully blurred image.
• Advantage for Dynamic Scenes: Useful for capturing depth information of moving objects, as all focus distances are captured simultaneously in time, avoiding misalignment issues in a traditional sequential focal stack.