Multi-Resolution Hash Grid

The structure uses a spatial hash function to map 3D grid vertex coordinates to indices in a fixed-size hash table. This allows the representation of a very large, fine-grained virtual grid using a small, bounded amount of memory. Collisions (multiple vertices mapping to the same hash entry) are handled gracefully by the subsequent neural network, which learns to disambiguate them during training.

• Key Benefit: Enables the use of extremely high-resolution grids (e.g., levels up to 2^19) without a memory explosion.
• Contrast: A traditional dense grid at equivalent resolution would be computationally infeasible.

The encoding is not a single grid but a composition of multiple grids at different resolutions (e.g., 16 levels from coarse to fine). Each level independently hashes its vertices and stores its own set of learnable feature vectors. The features from all levels are retrieved for a 3D point and concatenated before being passed to the tiny MLP.

• Purpose: Coarse levels capture broad scene geometry, while fine levels capture high-frequency texture and detail.
• Result: This mimics the behavior of Fourier features or positional encoding but in a learned, more parameter-efficient manner.

To query the encoding for a continuous 3D coordinate, the system performs trilinear interpolation within the voxel at each resolution level. It finds the eight surrounding grid vertices, looks up their feature vectors via the hash table, and interpolates between them.

• Effect: This provides spatial continuity (C^0 continuity), ensuring the input to the MLP changes smoothly as the query point moves, which is critical for stable gradient-based optimization.
• Efficiency: The interpolation is computationally cheap and fully differentiable.

The primary neural network in InstantNGP is a remarkably small MLP (often just 1-2 layers). Its role is to decode the concatenated, interpolated feature vectors from the hash tables into final density and color values.

• Design Rationale: The hash tables store the vast majority of the scene's learned information. The MLP acts as a lightweight decoder and collision resolver, learning a simple mapping from the high-dimensional feature space to output radiance.
• Impact: This drastically reduces the computational burden compared to the large MLPs used in original NeRF, enabling real-time performance.

Different resolution levels naturally adapt to the frequency content of the scene region. Smooth, uniform areas are sufficiently represented by coarser grid levels. Complex, textured surfaces activate the finer grid levels, which allocate hash entries to capture those details.

• Analogy: Similar to a mipmap or image pyramid, but the features are learned end-to-end.
• Practical Outcome: The model allocates its finite hash table capacity (parameters) adaptively to where detail is needed in the 3D space, improving parameter efficiency.

InstantNGP implements the entire pipeline—hashing, interpolation, and MLP evaluation—as fully-fused CUDA kernels. This means the computation for a batch of sampled points avoids costly memory transfers between CPU/GPU and intermediate buffers.

• Critical Optimization: The hash table lookups and small MLP operations are executed in parallel on the GPU with minimal overhead.
• Result: This kernel fusion is a primary reason InstantNGP achieves training times of seconds to minutes and rendering at interactive frame rates, whereas original NeRF took hours to train.

A multi-resolution hash grid is a learnable feature encoding that stores vectors at the vertices of multiple, independent grids at different resolutions. It uses spatial hashing to map 3D coordinates to a fixed-size hash table, enabling compact storage and O(1) lookup time.

• Key Innovation: Replaces the large, dense MLP in a standard NeRF with a small MLP that operates on features retrieved from this efficient grid.
• Function: For a given 3D coordinate, the feature vectors from the eight surrounding vertices at each resolution level are trilinearly interpolated and concatenated. This multi-scale representation is then passed to a tiny MLP to predict density and color.

Spatial hashing is the technique that makes the grid memory-efficient. A hash function maps 3D grid vertex indices to an entry in a fixed-size table.

• Collisions: Multiple vertices can hash to the same table entry. This is a deliberate design choice, not a bug. The model learns to handle these collisions through gradient descent.
• Trade-off: Controlled collisions act as a form of lossy compression, trading a minimal, often imperceptible, quality loss for a massive reduction in memory from gigabytes to megabytes.

The grid is not a single 3D lattice but a set of L levels (e.g., 16), each with a different resolution. Coarse levels capture broad scene geometry, while fine levels encode high-frequency texture details.

• Feature Concatenation: Features from all L levels are retrieved and concatenated, giving the MLP both local and contextual information.
• Progressive Detail: During training, the model effectively focuses on coarse-to-fine details as optimization progresses, similar to a positional encoding but with learned features.

Efficient implementation requires custom CUDA kernels. The fully-fused MLP and hash table lookups are optimized for parallel execution on GPUs.

• Core Operations: Parallel trilinear interpolation and hash table lookup across millions of sampled points per batch.
• Frameworks: The original InstantNGP uses a custom CUDA/C++ framework. Open-source re-implementations exist in PyTorch (e.g., torch-ngp) and JAX, providing more accessible APIs for researchers.

The hash grid is part of a family of techniques designed to speed up coordinate-based networks.

• TensorRF: Factorizes the 4D radiance field into compact vector-matrix tensors. More parameter-efficient but can be less performant than hashing for very high resolutions.
• Plenoxels: An explicit, non-neural voxel grid storing spherical harmonics. Renders quickly but requires significant memory and lacks the compactness of a learned hash table.
• Frequency (POSENC) vs. Grid Encodings: The hash grid provides a learnable alternative to fixed sinusoidal positional encoding, offering better performance with fewer MLP parameters.

InstantNGP is the framework that introduced the multi-resolution hash grid. It combines this data structure with a tiny Multi-Layer Perceptron (MLP) to achieve training times of seconds to minutes, a dramatic acceleration over original NeRF. The key innovation is using spatial hashing to store learnable feature vectors, enabling high-fidelity scene reconstruction with compact memory usage and fast, cache-coherent lookups.

Hash table encoding is the mechanism at the heart of the multi-resolution grid. Instead of storing a feature vector at every possible grid vertex (which would be memory-prohibitive), it uses a hash function to map spatial coordinates to a fixed-size table. This allows for:

• O(1) lookup time for feature retrieval.
• Collision handling where multiple vertices share a table entry, with gradients averaged during training.
• Compact storage independent of the grid's apparent resolution, enabling the use of many high-resolution levels.

Positional encoding is a precursor technique that maps low-dimensional input coordinates (like 3D position (x,y,z)) to a higher-dimensional space using a set of sinusoidal functions. This helps a standard MLP learn high-frequency details in color and geometry. The multi-resolution hash grid replaces explicit positional encoding; the learnable feature vectors retrieved from the hash tables provide a more flexible and parameter-efficient way for the MLP to represent high-frequency scene content.

Trilinear interpolation is the method used to query the multi-resolution hash grid at continuous 3D coordinates. For a given query point, the system:

1. Identifies the eight vertices of the axis-aligned grid cell containing the point.
2. Retrieves the feature vector for each vertex from the hash table.
3. Blends these eight vectors based on the point's relative position within the cell. This provides a continuous and differentiable signal from the otherwise discrete grid, allowing gradient-based optimization via backpropagation.

A feature vector is a small, dense array of floating-point numbers (e.g., 2-16 dimensions) stored at each entry in the multi-resolution hash table. These vectors are learnable parameters optimized during training. For a given 3D point, the retrieved and interpolated feature vectors from multiple grid resolutions are concatenated and fed into the tiny MLP. The MLP's role is simplified to decoding this rich, multi-scale feature representation into final color and density values.

This distinction clarifies the role of the hash grid. NeRFs traditionally use a purely implicit neural representation, where scene knowledge is stored in the weights of a large MLP. The multi-resolution hash grid introduces an explicit, structured component—a set of feature grids. This hybrid approach combines the benefits of explicit methods (fast lookup, local generalization) with the flexibility of implicit methods (continuous representation), leading to the speed and quality breakthroughs seen in InstantNGP.