5D Neural Field

The function's input is a 5D coordinate:

• 3D Spatial Location (x, y, z): Defines a point in the scene's volume.
• 2D Viewing Direction (θ, φ): Defines the spherical angles of the outgoing ray of light. This directional dependence is crucial for modeling view-dependent effects like specular highlights and reflections, which change based on the observer's position.

For any 5D query, the neural field outputs two key volumetric properties:

• Volume Density (σ): A scalar representing the differential probability of a ray terminating at that 3D point. This defines the scene's geometry and opacity.
• Color (RGB): The radiance (color) emitted from that point in the specified viewing direction. The network learns a continuous radiance field that fills the entire 3D space.

The scene is not stored as explicit geometry (meshes, point clouds) but is implicitly encoded within the weights of a neural network, typically a Multi-Layer Perceptron (MLP). This representation is:

• Compact: A scene can be represented by just the network's weights.
• Continuous and Infinitely Resolvable: It can be queried at any spatial resolution without predefined voxel grids.
• Differentiable: Enables end-to-end optimization from 2D images via differentiable volume rendering.

The primary application is photorealistic novel view synthesis. To render an image:

1. Ray Casting: Cast a ray through each pixel of a virtual camera.
2. Ray Marching: Sample multiple 5D points along each ray.
3. Neural Query: Evaluate the neural field at each sample.
4. Volume Rendering: Integrate the sampled colors and densities along the ray using the volume rendering equation to compute the final pixel color.

Standard MLPs are biased towards learning low-frequency functions. To capture high-frequency details in textures and geometry, the 3D coordinates (x,y,z) are first mapped to a higher-dimensional space using sinusoidal positional encoding. This transformation allows the MLP to approximate the plenoptic function more effectively, leading to sharp, detailed reconstructions.

A 5D neural field is a practical, learned approximation of the theoretical 7D plenoptic function, which describes all light rays (radiance) in a scene. The NeRF formulation makes two key simplifications:

• Wavelength is reduced to RGB.
• Time is held constant (for static scenes). Thus, it models the 5D plenoptic function of a static scene, capturing the full visual appearance from any viewpoint.

The plenoptic function is the theoretical 7D function describing the intensity of light observed from every 3D spatial position (x, y, z), in every 2D viewing direction (θ, φ), for every wavelength (λ), at every time (t). A 5D neural field is a practical, learnable approximation of this function, modeling radiance for static scenes at a single moment in time. It forms the mathematical basis for view synthesis and light field reconstruction.

An implicit neural representation encodes a continuous signal (like a 3D shape, sound, or image) as the weights of a neural network that acts as a function. A 5D neural field is a specific type of implicit representation where the network maps 5D coordinates (3D location + 2D direction) to scene properties. This contrasts with explicit representations like meshes or voxel grids, offering infinite resolution and memory efficiency but requiring querying the network for every point.

Volume rendering is the computer graphics algorithm used to generate a 2D image from a 3D volumetric field. In a NeRF, the 5D neural field outputs volume density (σ) and radiance (RGB) at sampled 3D points. The classic volume rendering integral accumulates this data along each camera ray to compute the final pixel color. This process is differentiable, enabling end-to-end training from 2D images via photometric loss.

Positional encoding (or Fourier feature mapping) is a critical preprocessing step for 5D neural fields. Raw input coordinates (x, y, z, θ, φ) are mapped to a higher-dimensional space using a set of sinusoidal functions: γ(p) = [sin(2^0 π p), cos(2^0 π p), ..., sin(2^(L-1) π p), cos(2^(L-1) π p)]. This allows a standard Multi-Layer Perceptron (MLP) to effectively learn high-frequency details in color and geometry, which it struggles with using low-dimensional inputs alone.

A 4D neural field extends the concept by incorporating a temporal dimension, mapping a 3D spatial coordinate and a time value to scene properties. It is the core representation for Dynamic NeRF models that capture scenes with motion, deformation, or other changes over time. This is often implemented by adding a time input to the network or by learning a deformation field that maps coordinates from a canonical space to each observed time step.

A radiance field is the component of a 5D neural field that specifically defines the color (radiance) of light emitted in every direction at every point in space. It is often modeled as a function c(x, d) where x is 3D position and d is the 2D viewing direction. When coupled with a density field σ(x), it forms the complete volumetric scene description used for photorealistic rendering. The term is sometimes used interchangeably with 5D neural field.