Ray Marching

The core mathematical primitive for most ray marching. An SDF is a function f(p) that returns the shortest distance from a point p in space to the surface of an object, with the sign indicating whether the point is inside (negative) or outside (positive).

• Key Property: The magnitude of the gradient is 1 (||∇f(p)|| ≈ 1), enabling stable, efficient stepping.
• Use Case: Defining smooth, precise geometry for constructive solid geometry (CSG), soft shadows, and ambient occlusion.
• Example: The SDF for a sphere centered at c with radius r is length(p - c) - r.

The most common and efficient ray marching algorithm for SDFs. Instead of taking fixed-size steps, the ray advances by the exact distance to the nearest surface at each sample point, guaranteed not to overshoot.

• Algorithm: For a ray origin o and direction d, iterate: p = o + t * d; distance = sdf(p); t += distance; until distance < epsilon or t > max_distance.
• Benefit: Minimizes the number of steps required to find a surface intersection.
• Limitation: Requires a proper SDF; performance degrades with complex, high-frequency details.

The method used in neural rendering (e.g., NeRF) where a ray samples a continuous density or occupancy field.

• Process: The ray is partitioned into segments. At each sample point, a neural network outputs a density σ and color c.
• Rendering Equation: Colors and densities are composited using volume rendering to compute the final pixel color: C = Σ T_i * (1 - exp(-σ_i * δ_i)) * c_i, where T_i = exp(-Σ_{j=1}^{i-1} σ_j * δ_j) is transmittance.
• Contrast with SDFs: No explicit surface; appearance emerges from semi-transparent volumetric media.

Optimization strategies to avoid sampling empty or homogeneous space, critical for real-time performance.

• Coarse-to-Fine Sampling: A lightweight proposal network first samples the ray coarsely to predict an importance distribution, guiding a second network to sample densely only in relevant regions.
• Empty Space Skipping: Using data structures like bounding volume hierarchies (BVH), spatial hash grids, or occupancy grids to quickly skip large empty intervals.
• Example: Instant NGP uses a multi-resolution hash grid to enable near-instant lookup of spatial features, bypassing the need for deep network evaluation in empty space.

Ray marching naturally simulates complex light transport phenomena by casting secondary rays from sample points.

• Soft Shadows: March a ray from a surface point towards a light source. The percentage of the march that is occluded (density > 0) determines shadow softness.
• Ambient Occlusion: Approximated by marching rays in a hemisphere around the surface normal and accumulating how much they are blocked by nearby geometry.
• Subsurface Scattering: Modeled by sampling and blending density and color along refracted rays inside a volumetric material.
• Benefit: Unifies geometry and lighting calculations in a single sampling framework.

A pivotal characteristic for modern neural graphics. The ray marching process can be made end-to-end differentiable, enabling gradient-based optimization of scene parameters.

• Core Mechanism: The volume rendering integral is differentiable with respect to the sampled densities and colors. This allows gradients to flow back through the sampling positions to the underlying neural field parameters.
• Application: This is the foundation for Neural Radiance Field (NeRF) training, where a multilayer perceptron (MLP) is optimized from 2D images without 3D supervision.
• Challenge: Requires careful implementation to maintain numerical stability through the iterative sampling process.

Ray marching is the definitive technique for rendering volumetric phenomena where light interacts with a participating medium. This includes:

• Fog, smoke, and clouds: Modeling light absorption and scattering through heterogeneous densities.
• Fire and plasma: Simulating emissive, self-illuminating volumes with complex internal structure.
• Subsurface scattering: Approximating light transport within translucent materials like skin, wax, or marble by treating them as volumetric slabs.

The technique excels here because it can sample a density field at any point in space, calculating accumulated transmittance and in-scattering along the ray path, which is infeasible with traditional polygon rasterization.

This is a foundational application for real-time graphics. Ray marching is used to render scenes defined by Signed Distance Functions (SDFs), which mathematically define a shape's volume.

Key advantages:

• Infinite detail: SDFs are procedural and resolution-independent, enabling perfectly smooth surfaces at any zoom level.
• Boolean operations: Shapes can be combined (union, intersection, subtraction) trivially via SDF math.
• Efficient intersection: The SDF value provides a safe marching distance, allowing the ray to take large, adaptive steps through empty space (Sphere Tracing).

This is central to ShaderToy-style graphics, CAD visualization of constructive solid geometry, and rendering neural implicit surfaces like those from an occupancy network.

Ray marching is the core rendering engine for Neural Radiance Fields (NeRFs) and related implicit neural representations. A NeRF encodes a scene within the weights of a multilayer perceptron (MLP) that outputs density and view-dependent color for any 3D coordinate.

The rendering process:

1. A ray is cast from the camera through each pixel.
2. The ray is marched, sampling 3D points along its path.
3. For each point, the NeRF MLP is queried for density (σ) and RGB color.
4. Colors and densities are composited using volume rendering equations to produce the final pixel color.

This allows photorealistic novel view synthesis from sparse 2D images, powering applications in VR, content creation, and digital twins.

Ray marching is indispensable for visualizing continuous 3D scalar fields from scientific and medical imaging data.

Common data sources:

• CT and MRI scans: Volumetric data representing tissue density or proton concentration.
• Computational fluid dynamics (CFD): Simulated fields of pressure, temperature, or velocity.
• Astrophysical simulations: Density distributions of gas clouds or dark matter.

The technique allows researchers to apply transfer functions that map data values (e.g., Hounsfield units) to color and opacity. By compositing these values along the ray, it creates informative direct volume renderings that reveal internal structures—such as a tumor in a scan or a vortex in a fluid—without extracting an explicit surface mesh.

Ray marching is the only practical method for rendering 3D fractals like the Mandelbulb, Mandelbox, and Julia sets in real time. These structures are defined by iterative mathematical functions where the distance to the surface (the SDF) can be approximated but the surface itself is infinitely complex and cannot be polygonized.

The rendering workflow:

1. For each point in space, an escape-time algorithm calculates an approximate distance to the fractal set.
2. This distance estimate serves as the SDF for the sphere tracing variant of ray marching.
3. The ray progresses until it hits the surface or exceeds a maximum step count.

This enables the exploration of infinitely detailed, mathematically generated landscapes that are iconic in demoscene and algorithmic art.

Ray marching is used as a sampling technique to approximate complex lighting effects within a unified framework.

• Screen-Space Ambient Occlusion (SSAO): Although often a screen-space hack, higher-quality SSAO implementations may use a limited form of ray marching within the depth buffer to estimate how occluded a point is by its surroundings.
• Volumetric Global Illumination: For participating media, ray marching directly calculates in-scattering from light sources, producing effects like god rays (crepuscular rays) where light shafts become visible through volumetric fog.
• Signed Distance Field (SDF) Soft Shadows: When ray marching an SDF scene, shadows can be computed by marching a ray from the surface point toward the light source. The minimum SDF value encountered along this light ray provides a measure of penumbra size, enabling soft, realistic shadows with a single sample.