Ray Sampling

The foundational sampling strategy where points are placed at uniform intervals along a ray within predefined near and far bounds. This is the baseline used in the original NeRF paper.

• Mechanism: The ray segment is divided into N equally spaced bins, and one sample is taken randomly within each bin to provide some stochastic coverage.
• Purpose: Provides a simple, unbiased initial estimate of the scene's geometry.
• Limitation: It is computationally inefficient, as many samples are wasted in empty space or regions of homogeneous density.

A two-stage, importance sampling technique introduced in the original NeRF to allocate samples efficiently based on learned scene geometry.

• Coarse-to-Fine: A 'coarse' network is first evaluated with uniform samples to produce a piecewise-constant PDF (Probability Density Function) along the ray.
• Informed Sampling: This PDF is then used to draw a second set of samples for the 'fine' network, concentrating them in regions with high predicted density (i.e., near surfaces).
• Impact: This strategy dramatically improves rendering quality without a proportional increase in the total number of samples, as computation is focused where it matters most.

The mathematical procedure used to draw samples from a non-uniform distribution, such as the PDF produced by the coarse network in hierarchical sampling.

• Process: The PDF is integrated to create a Cumulative Distribution Function (CDF). Samples are then drawn by generating uniform random numbers and finding the corresponding ray depth via the inverse of the CDF.
• Role in NeRF: This is the engine that enables importance sampling, transforming a learned density distribution into specific 3D point locations for the fine network to query.

The rendering algorithm that converts discrete point samples into a final pixel color. It is the practical implementation of the volume rendering equation.

• Alpha Compositing: At each sampled point, the network outputs density (σ) and color (c). These are converted into transmittance and alpha values.
• Accumulation: Colors from all samples are blended front-to-back using the classic over operator, simulating light absorption and emission along the ray.
• Core Equation: The final pixel color is a weighted sum of all sample colors, where weights are a function of transmittance and density.

An advanced, learned sampling paradigm used in accelerated NeRF variants like Mip-NeRF 360 and InstantNGP to further optimize sample placement.

• Mechanism: One or small, lightweight proposal MLPs predict histograms of density. These are not used for final color, but only to generate a sampling distribution.
• Iterative Refinement: Rays are sampled multiple times (e.g., 2-3 passes), with each proposal network using samples from the previous stage to predict a better distribution for the next.
• Benefit: Decouples sample placement from the final radiance field, allowing the main network to be evaluated on a minimal set of highly relevant points, enabling real-time performance.

A sampling strategy that moves from point samples to conical frustums to achieve built-in anti-aliasing.

• Problem: Standard NeRF samples infinitesimal points, causing aliasing artifacts when training and testing at different image resolutions.
• Solution: Instead of a point, Mip-NeRF considers the integrated volume of a conical frustum defined by a pixel's footprint. The positional encoding is applied to the statistical mean and variance of this region.
• Outcome: The network learns a pre-filtered representation, producing crisp renders at any scale without high-frequency 'jaggies' or blurring.

The foundational computer graphics technique used by NeRF to generate a 2D image. It involves integrating optical properties—color and volume density—along rays cast from the camera through the 3D scene. The NeRF model implements a continuous version of this integral using numerical quadrature.

• Core Equation: The rendered color for a pixel is the weighted sum of colors sampled along its corresponding ray.
• Alpha Compositing: The integration naturally handles transparency and occlusion through the accumulated density.

The specific numerical algorithm used to approximate the volume rendering integral in NeRF. It is the practical implementation of ray sampling.

• Process: The algorithm marches along each camera ray, querying the neural network at discrete 3D points.
• Stratified Sampling: Points are initially sampled uniformly along the ray to ensure broad coverage.
• Hierarchical Refinement: A second, more focused set of samples is taken based on the density distribution from the first pass, concentrating computation where it matters most.

A two-stage, importance sampling strategy introduced in the original NeRF paper to improve efficiency. It prevents wasted computation in empty space.

• Coarse Network: A first pass samples points and predicts a piecewise-constant density function.
• Fine Sampling: This density function is used to draw a second set of samples, biased towards regions likely to contain scene content.
• Result: The final rendering uses a combined set of samples from both passes, dramatically improving detail with minimal additional queries.

The continuous function that a NeRF learns to represent. Ray sampling is the process of querying this function.

• Input: A 3D spatial coordinate (x, y, z) and a 2D viewing direction (θ, φ).
• Output: A volume density (σ) and a view-dependent RGB color.
• Continuous Representation: Unlike a voxel grid or mesh, this field is defined at every point in space, enabling theoretically infinite resolution.

An advanced NeRF variant that addresses a critical flaw in basic ray sampling: treating rays as infinitesimally thin lines, which causes aliasing.

• Conical Frustums: Instead of sampling points along a ray, Mip-NeRF considers the volume of a cone representing a pixel's footprint.
• Integrated Positional Encoding (IPE): It calculates the expected positional encoding over this volume, allowing the model to reason about scale.
• Outcome: This produces anti-aliased renderings that remain sharp when zoomed in and smooth when zoomed out, matching the behavior of a physical camera.

A pre-processing transformation applied to 3D coordinates before they are fed into the NeRF's MLP. It is essential for the network to represent high-frequency scene details.

• Function: Maps low-dimensional inputs to a higher-dimensional space using a set of sinusoidal functions: γ(p) = [sin(2^0 π p), cos(2^0 π p), ..., sin(2^(L-1) π p), cos(2^(L-1) π p)].
• Impact: Without it, the MLP tends to produce overly smooth, blurry outputs—a phenomenon known as spectral bias. Ray sampling fetches these encoded coordinates.