Volume Rendering

The Emission Term represents light that is generated or emitted from within the volume itself at a given point. It is a source term in the equation, adding radiance to the ray.

• Mathematical Form: Typically denoted as C(s)σₐ(s), where C(s) is the color (radiance) emitted and σₐ(s) is the absorption coefficient at point s.
• Physical Analogy: Similar to a light bulb or a glowing gas cloud within the volume.
• In NeRF: The neural network directly outputs an RGB color at each sampled 3D point, which acts as the emitted radiance for that infinitesimal particle.

Absorption and Out-Scattering are attenuation processes that reduce the intensity of light traveling along a ray. They are combined into the Extinction Coefficient, σₜ(s) = σₐ(s) + σₛ(s).

• Absorption (σₐ): Photons are absorbed by the medium and converted to other forms of energy (e.g., heat).
• Out-Scattering (σₛ): Photons are deflected into different directions, away from the ray path.
• The Role of σ: In the VRE, the exponential decay term exp(-∫σₜ(t)dt) models how much light from behind a point reaches it. In NeRF, the network outputs a volume density σ, which directly serves as this extinction coefficient.

Transmittance, T(s), is the probability that a photon travels from the start of the ray to a point s without being absorbed or scattered away. It quantifies the cumulative attenuation up to that point.

• Mathematical Definition: T(s) = exp( -∫₀ˢ σₜ(t) dt ).
• Physical Meaning: It acts as a 'visibility' multiplier. Light contributions from deeper in the volume are weighted by how much of the preceding medium they had to pass through.
• In Rendering: This is computed numerically via alpha compositing. The accumulated opacity α = 1 - exp(-σ δ), where δ is the distance between samples, is used to build up transmittance step-by-step.

In-Scattering is the process where light from other directions is scattered into the ray path. It is governed by the Phase Function, p(ω → ω'), which describes the angular distribution of scattered light.

• Key Property: The phase function is normalized; its integral over all directions is 1.
• Isotropic vs. Anisotropic: A simple isotropic phase function scatters light equally in all directions. Real-world media (like fog or skin) use anisotropic functions (e.g., Henyey-Greenstein) to model forward or backward scattering.
• In NeRF: Standard NeRF models a vacuum with no in-scattering. It assumes light travels directly from the light source to a point and is then emitted toward the camera. Extensions like NeRF in participating media explicitly model this term.

The full Volume Rendering Equation integrates all components along a ray from the camera tₙ to tₓ. The final color C(r) for ray r is:

C(r) = ∫ₜₙᵗᶠ T(t) [ Cₑ(t) + ∫₄π p(ω→ω') Lᵢ(t,ω') dω' ] dt

Where:

• T(t): Transmittance to point t.
• Cₑ(t): Emitted radiance at t.
• ∫₄π p(...)Lᵢ(...) dω': The in-scattering integral over all incoming directions ω'.
• The NeRF Simplification: NeRF assumes no external illumination or in-scattering. The equation reduces to C(r) = ∫ T(t) σ(t) c(t) dt, where σ(t) is density and c(t) is the view-dependent emitted color.

Solving the VRE's continuous integral requires numerical quadrature, implemented via Ray Marching. The ray is discretized into N samples, and the integral is approximated as a weighted sum.

• The Discrete Equation: Ĉ(r) = Σᵢ₌₁ᴺ Tᵢ (1 - exp(-σᵢ δᵢ)) cᵢ, where Tᵢ = exp(-Σⱼ₌₁ⁱ⁻¹ σⱼ δⱼ).
• Alpha Compositing: The term (1 - exp(-σᵢ δᵢ)) is the alpha value αᵢ for the i-th sample. The final pixel color is computed via front-to-back alpha blending:
  • C ← C + T * α * c
  • T ← T * (1 - α)
• Hierarchical Sampling: To sample efficiently, NeRF uses a two-stage process: a 'coarse' network informs a 'fine' network where to place more samples in regions of high density (σ).

Ray marching is the core numerical integration algorithm used in volume rendering. It approximates the volume rendering integral by sampling a 3D field at discrete points along a ray from the camera.

• Process: Steps along a ray, querying the volume's density and color at each sample point.
• Accumulation: Uses the alpha compositing equation to blend samples from back to front, simulating light absorption and emission.
• NeRF Application: In a Neural Radiance Field, ray marching queries a neural network at each 3D point to get the radiance and density, which are then composited to produce a pixel color.

The volume rendering equation is the continuous mathematical model that ray marching approximates. It computes the color C of a pixel by integrating light contributions along a ray r(t) from the near bound t_n to the far bound t_f.

The standard form is: C(r) = ∫_tn^tf T(t) * σ(r(t)) * c(r(t), d) dt where:

• σ(r(t)) is the volume density at point t.
• c(r(t), d) is the radiance (color) emitted at that point in direction d.
• T(t) = exp(-∫_tn^t σ(r(s)) ds) is the transmittance, representing the probability the ray travels from t_n to t without hitting any particles. This equation is the theoretical foundation for all volumetric rendering, including NeRF.

Alpha compositing is the discrete, front-to-back algorithm used to approximate the volume rendering integral during ray marching. It is also known as the over operator.

For N ordered samples along a ray, the accumulated color C and opacity α are computed iteratively:

• C_i = C_i-1 + (1 - α_i-1) * α_i * c_i
• α_i = α_i-1 + (1 - α_i-1) * α_i

Key Properties:

• c_i is the color of the i-th sample.
• α_i is the opacity of the i-th sample, derived from density (α_i = 1 - exp(-σ_i * δ_i), where δ_i is the distance between samples).
• The term (1 - α_i-1) is the remaining transmittance, ensuring deeper samples contribute less if the ray is already occluded.

A transfer function is a user-defined mapping that assigns optical properties (color and opacity) to the scalar values in a volumetric dataset. It is a critical tool for visualizing scientific and medical data (e.g., CT, MRI scans).

• Purpose: Makes specific tissues or materials visible by mapping data values to high opacity and distinct colors.
• Contrast with NeRF: In a traditional volume renderer, the transfer function is applied to pre-existing 3D data. In NeRF, the neural network learns an implicit transfer function, outputting density σ and color c directly from 3D coordinates.

The emission-absorption model is the specific physical light interaction model assumed by the classic volume rendering equation. It describes how light behaves as it passes through a participating medium.

• Absorption: Light is attenuated as it passes through the volume. The probability of absorption is defined by the absorption coefficient, related to density σ.
• Emission: The volume itself can emit light (e.g., a hot gas, a neon cloud). This emitted radiance is the c(r(t), d) term.
• Key Assumption: The model typically does not account for complex in-scattering (light scattering into the ray from other directions), which would require the full radiative transfer equation. NeRF and most real-time volume rendering use this simpler, effective model.

Differentiable rendering is a framework that allows gradients to flow from a synthesized 2D image back to the underlying 3D scene parameters. This is the enabling technology for training Neural Radiance Fields.

• Core Mechanism: Makes the rendering process (ray marching, sampling, compositing) differentiable with respect to scene parameters like density and color.
• NeRF Training: The photometric loss between a rendered image and a ground truth image generates gradients. These gradients propagate backward through the volume rendering pipeline to update the weights of the neural network defining the radiance field.
• Impact: It turns rendering from a final visualization step into an optimization tool for reconstructing 3D geometry and appearance from 2D images.