G-Buffer

The World Position buffer stores the 3D coordinates (x, y, z) of each visible surface point in world space for every pixel. This is the foundational geometric data, often reconstructed from depth, used for lighting calculations, spatial queries, and as input to neural networks in deferred neural rendering.

• Primary Use: Lighting (distance/position-based), screen-space reflections, and world-space reconstruction for neural post-processing.
• Storage: Typically three 32-bit floating-point channels (RGB).

The Surface Normal buffer stores the orientation vector of each surface, typically in world or view space. This vector is perpendicular to the surface and is critical for calculating how light reflects, which defines shading, specular highlights, and material appearance.

• Primary Use: Diffuse and specular lighting calculations (dot product with light direction).
• Storage: Three normalized 8-bit or 16-bit channels. Often encoded as two components (e.g., octahedral encoding) to save bandwidth.

The Albedo buffer stores the base, intrinsic color of a surface, excluding lighting and shadow information. It represents the percentage of incident light that is reflected diffusely. This is the "clean" color data that lighting calculations operate on.

• Primary Use: Multiplying with direct and indirect light contributions to produce the final shaded color.
• Key Distinction: Must be separated from baked lighting or shadows to allow for dynamic relighting.

The Depth buffer (or Z-buffer) stores the distance from the camera to the closest visible surface for each pixel. It is the most compact representation of scene geometry and is used for visibility testing, reconstructing world position, and managing rendering order.

• Primary Use: Occlusion culling, world position reconstruction, and post-processing effects like depth-of-field.
• Technical Note: Often stored in a non-linear, projection-space format to maximize precision for nearby objects.

This buffer encodes surface material properties. A Material ID is an integer index pointing to a material definition. More commonly, a PBR (Physically Based Rendering) buffer stores continuous values like Roughness (microsurface smoothness) and Metallic (dielectric vs. conductor).

• Primary Use: Dictating how the shading model (e.g., Cook-Torrance BRDF) behaves per pixel.
• Example: A pixel with high roughness scatters light broadly (matte), while high metallic uses the albedo as the reflectance color.

Advanced G-buffers include channels for Emissive color (for light-emitting surfaces), Ambient Occlusion (baked cavity shadows), and Subsurface Scattering data. In deferred neural rendering, this can be extended to store Neural Features—learned latent codes from a neural texture that a subsequent network decodes into final appearance.

• Primary Use: Adding complex, view-dependent effects and enabling neural post-processing without re-rasterizing geometry.
• Connection: This is the bridge between traditional rasterization and neural rendering techniques like deferred neural rendering.

The original and most common application. The G-buffer is rasterized in a first geometry pass. A second lighting pass then uses the stored world position, normals, and albedo to compute complex lighting—including numerous dynamic lights—without re-rasterizing geometry for each light source. This decouples geometric complexity from lighting complexity.

• Key Benefit: Enables hundreds of lights in a scene with minimal performance cost.
• Trade-off: Increased memory bandwidth and inability to use hardware anti-aliasing like MSAA.

A modern evolution where a neural network replaces the traditional lighting/shading pass. The G-buffer (containing data like depth, normals, and roughness) is fed as input channels to a small neural network (e.g., a convolutional U-Net) that outputs the final shaded pixel. This allows:

• Learning complex materials and global illumination from data.
• Real-time style transfer and post-processing effects.
• Filling in missing information (e.g., from low-sample ray tracing) via learned denoising.

Frameworks like NVIDIA's Real-Time Denoisers (NRD) and research in neural supersampling (DLSS) use this principle.

G-buffer data is ideal for high-quality, post-process effects computed in 2D screen space.

• Screen-Space Reflections (SSR): Uses world position and normals to ray march within the depth buffer for realistic local reflections.
• Screen-Space Ambient Occlusion (SSAO): Samples the depth buffer around a pixel to approximate ambient light occlusion.
• Decal & Projection Systems: Precisely project textures onto complex geometry using depth and normal buffers for alignment.
• Edge Detection & Outlines: Uses the normal buffer to detect object silhouettes for non-photorealistic rendering.

By storing the previous frame's world position for each pixel, the GPU can compute per-pixel motion vectors. These 2D vectors describe how each surface point moved between frames. This is critical for:

• Temporal Anti-Aliasing (TAA): Rejects flickering and reduces aliasing by accumulating samples over time.
• Temporal Upsampling/Super-Resolution: Reconstructs a high-resolution frame from multiple low-resolution frames (e.g., FSR 2, TAAU).
• Motion Blur: Applying accurate, per-object motion blur in post-processing.
• Asynchronous Reprojection in VR: Warping previous frames to match new head poses to reduce latency.

G-buffers are a powerful tool for generating perfectly labeled synthetic datasets for computer vision model training. A single render can produce multiple perfectly aligned ground truth channels:

• Semantic Segmentation Maps: From material IDs.
• Instance Segmentation Maps: From object IDs.
• Depth Maps: For monocular depth estimation training.
• Surface Normal Maps: For geometry estimation tasks.
• Optical Flow Ground Truth: From computed motion vectors.

This enables training of robust perception models (e.g., for autonomous systems) without costly manual labeling, using engines like Unreal Engine or Unity.

The explicit separation of surface properties allows for non-destructive, highly controlled final image assembly.

• Relighting & Material Editing: Artists can adjust lighting or swap albedo/material properties in post-production without re-rendering the entire scene.
• Volumetric Effects Integration: Depth and world position allow fog, light shafts, and participating media to be correctly integrated with opaque geometry.
• ID-Based Filtering: Using stored object or material IDs to apply selective effects, color grading, or blur to specific scene elements.
• Hybrid Renderer Integration: G-buffers provide a common interface for combining rasterized elements with ray-traced elements, directing rays only where needed.

Deferred Neural Rendering is a modern graphics pipeline that replaces the traditional deferred shading's lighting pass with a neural network. The G-buffer serves as the structured input to a neural renderer (e.g., a small MLP or CNN), which outputs the final image. This enables photorealistic effects like complex global illumination, denoising, and neural textures that are difficult to model with traditional shaders.

Motion Vectors are a per-pixel 2D vector stored in a G-buffer channel. They encode the screen-space displacement of a surface point between consecutive frames. This data is critical for:

• Temporal Anti-Aliasing (TAA): Reprojecting samples from previous frames.
• Temporal Super-Resolution: Accumulating data across frames.
• Motion Blur: Simulating camera or object motion. Accurate motion vectors are essential for temporal stability in real-time rendering.

A Material ID or Object ID is an integer index stored per-pixel in the G-buffer. It acts as a unique identifier for:

• Post-Processing: Applying material-specific filters (e.g., different blur to skin vs. metal).
• Debugging: Visualizing different materials or objects in the scene.
• AI/ML Tasks: Providing semantic ground truth for training computer vision models directly from the rendering pipeline. It enables precise, pixel-perfect selection and masking.

These are advanced optimizations for deferred shading with many light sources.

• Tiled Deferred: The screen is divided into tiles (e.g., 16x16 pixels). Per tile, a compute shader culls lights against the tile's depth bounds (from the G-buffer), creating a light list for efficient shading.
• Clustered Deferred: Extends this to 3D, dividing view frustum into clusters (X, Y, and depth). This handles light range and distribution in 3D space more accurately than tiled methods, especially for large scenes.