Importance Sampling

The primary goal of importance sampling is to reduce the variance of a Monte Carlo estimator. By drawing samples from a proposal distribution $q(x)$ that is proportional to the integrand $f(x)$, the estimator's variance can be driven toward zero. The optimal proposal is $q^*(x) = |f(x)| / \int |f(x)| dx$, which yields zero variance. In practice, a close approximation to this optimal distribution dramatically lowers noise compared to uniform sampling.

The proposal distribution $q(x)$ is the engine of importance sampling. It must satisfy two key properties:

• Full Support: It must be nonzero wherever the target function $f(x)$ is nonzero ($q(x) > 0$ when $f(x) \neq 0$).
• Known Density: Its probability density function (PDF) must be known and, ideally, easy to sample from. The estimator is reweighted by the importance weight $w(x) = p(x)/q(x)$, where $p(x)$ is the original sampling distribution. A poor proposal (e.g., one that misses high-value regions) can increase variance, making its design critical.

In Neural Radiance Fields (NeRF) and related methods, importance sampling is used to guide ray marching. Instead of sampling a ray uniformly, samples are concentrated in regions of high volume density where light interaction is likely.

• A coarse network or proposal network first predicts a piecewise-constant PDF along each ray.
• A fine network then samples more points according to this PDF, focusing computation on opaque surfaces and complex volumetric media. This hierarchical approach is essential for achieving photorealistic results with tractable compute, as it avoids wasting samples on empty space.

Importance sampling is the algorithmic core of efficient ray marching. The technique transforms the rendering integral: $C(r) = \int_{t_n}^{t_f} T(t) \sigma(r(t)) c(r(t), d) dt$ into a discrete sum where sample locations $t_i$ are drawn from a distribution informed by the predicted density $\sigma$. The transmittance $T(t)$ acts as a natural importance weight. Methods like coarse-to-fine sampling and the use of proposal networks in Instant NGP are direct implementations of importance sampling to allocate samples along rays where $T(t)\sigma(t)$ is largest.

Properly implemented importance sampling is an unbiased estimator—its expected value equals the true integral. However, practical approximations introduce trade-offs:

• Using a non-optimal proposal increases variance but maintains unbiasedness.
• Techniques like resampling or clipping large weights can reduce variance at the cost of introducing slight bias.
• In neural rendering, the proposal network is trained jointly with the radiance field, creating a feedback loop that optimizes the proposal to minimize final rendering loss (a form of variance reduction).

Two common estimators exist:

• Unnormalized Estimator: $\hat{I} = \frac{1}{N} \sum_{i=1}^N \frac{f(x_i)}{q(x_i)}$. Requires $q(x)$ to be a proper PDF.
• Self-Normalized Estimator: $\hat{I} = \sum_{i=1}^N w_i f(x_i) / \sum_{i=1}^N w_i$, where $w_i = p(x_i)/q(x_i)$. This is crucial when only an unnormalized proposal is available (common in adaptive methods). The self-normalized estimator is biased for finite N, but consistent (bias vanishes as $N \to \infty$). It is the standard form used in particle filters and adaptive rendering.

The foundational numerical method that importance sampling optimizes. Monte Carlo integration estimates an integral by averaging random samples. Its error decreases with the square root of the number of samples (O(1/√N)), making it computationally expensive for high-variance functions. Importance sampling is a variance reduction technique applied within this framework to achieve the same accuracy with far fewer samples.

• Core Principle: ∫ f(x) dx ≈ (1/N) Σ f(X_i) / p(X_i), where X_i ~ p(x).
• Baseline Method: Naive Monte Carlo uses a uniform distribution p(x).
• Application in Rendering: Used for computing the rendering equation, which sums light contributions from all possible paths.

A lightweight neural network that predicts an optimal sampling distribution for ray marching. In frameworks like Mip-NeRF 360, a proposal network is trained jointly with the main NeRF to output a piecewise-constant density along a ray. This density is used for hierarchical sampling, guiding the main network to sample more points in regions of high expected radiance (e.g., near surfaces).

• Function: Maps a ray and coarse scene features to a probability distribution.
• Benefit: Dramatically reduces the number of queries to the large, expensive main NeRF model.
• Implementation: Often trained using a loss that minimizes the discrepancy between its histogram and the main network's final weights.

A hierarchical, two-pass sampling strategy that implements importance sampling efficiently. First, a coarse network or proposal network evaluates a ray at a sparse set of locations to estimate a preliminary density distribution. Second, a fine network samples additional points biased towards regions of high density identified in the first pass. This allocates computation where it matters most.

• Process: 1. Uniform coarse sampling. 2. Importance-based fine sampling.
• Outcome: Higher fidelity rendering without a linear increase in compute.
• Variant: Some methods use more than two levels for progressively refined sampling.

The volumetric rendering algorithm that importance sampling accelerates. Ray marching steps along a camera ray through a 3D scene, querying a neural radiance field (density and color) at discrete intervals. The final pixel color is computed via alpha compositing. Naive uniform stepping is inefficient. Importance sampling concentrates these steps around surfaces where density is non-zero.

• Core Operation: C = Σ T_i * α_i * c_i, where T_i is transmittance, α_i is opacity, c_i is color.
• Sampling Challenge: Missing a high-density region leads to incorrect transparency or holes.
• Optimization: Adaptive step size based on local density gradient (e.g., using the NeRF volume rendering equation).

A complementary variance reduction technique often used before importance sampling. The domain (e.g., a ray segment) is divided into strata (sub-intervals), and one sample is taken uniformly within each stratum. This ensures samples are spread evenly across the entire domain, preventing clumping and reducing variance from the initial sampling pass.

• Benefit: Guarantees better coverage than purely random sampling.
• Standard Practice: The initial coarse pass in NeRF typically uses stratified sampling.
• Combination: Often used to generate the initial samples that a proposal network then refines.

An advanced technique that combines samples from multiple proposal distributions. In complex lighting scenarios (e.g., direct and indirect illumination), no single importance sampling distribution is optimal. MIS weights samples from different strategies (e.g., sampling the light source vs. sampling the BRDF) to minimize variance further. It is governed by the balance heuristic or power heuristic.

• Mathematical Foundation: ∫ f(x) dx ≈ Σ (1/N_i) Σ w_i(X_{i,j}) * f(X_{i,j}) / p_i(X_{i,j}).
• Application: Critical for path tracing and photorealistic offline rendering.
• Neural Rendering Link: Emerging research integrates MIS into real-time neural path tracing pipelines.