Temporal Anti-Aliasing (TAA)

The fundamental operation of TAA is reprojection. For each pixel in the current frame, the algorithm uses motion vectors to locate where that surface point was in the previous frame. The color from that location is fetched from a history buffer—a texture storing the anti-aliased result of the prior frame. This reprojected sample is then blended with the current frame's jittered sample.

• Motion Vectors: Per-pixel 2D vectors, typically rendered in a separate pass, that encode screen-space motion from camera and object movement.
• History Buffer Management: Critical for handling disocclusions (newly revealed areas) and rejecting invalid history. Techniques like neighborhood clamping (clipping the history sample to the color range of nearby current pixels) prevent ghosting artifacts.

To gather unique color information each frame, TAA applies a subpixel jitter to the camera's projection matrix. This offsets the entire rendered image by a fraction of a pixel (e.g., (0.5, -0.5) in a 2x2 pattern). Over a sequence of frames, these jittered samples 'fill in' the gaps between physical pixel centers.

• Sample Pattern: Uses a low-discrepancy sequence (like Halton or Hammersley) to ensure good sample distribution.
• Aliasing Transformation: Converts spatial aliasing (jagged edges) into temporal aliasing (flickering), which is then smoothed by the temporal accumulation process.
• Integration with Rendering: The same jitter offset must be used for all rendering passes (depth, motion vectors, shading) to maintain consistency.

The core smoothing filter of TAA is a recursive, exponential moving average. The final color for a pixel is a blend between the newly rendered, jittered sample and the reprojected history sample.

• Blend Factor (Alpha): Typically a small, fixed value (e.g., 0.1). A low alpha prioritizes history, providing strong smoothing but risking blurring; a high alpha reacts faster to change but provides less anti-aliasing.
• Exponential Moving Average: The influence of a given frame's sample decays exponentially over time. This provides effective noise reduction over multiple frames.
• Weighting: More advanced implementations use variance-based or luminance-based weighting to adjust the blend factor per pixel, preserving detail in static areas and reacting quickly in motion.

TAA's primary artifacts are ghosting (trails behind moving objects) and excessive blur. Mitigation strategies are essential for robust implementation.

• Neighborhood Clamping (AABB Clamp): The most common method. The history sample's color is clamped to the minimum and maximum color found in the current frame's 3x3 pixel neighborhood. This rejects history that is too dissimilar, eliminating most ghosting.
• Velocity-Based Rejection: History is fully rejected for pixels with very high motion vector magnitude.
• Sharpening Pass: A post-TAA sharpening filter (e.g., a contrast-adaptive unsharp mask) is often applied to counteract the natural softening from temporal blending.

TAA is not just for anti-aliasing; it's a foundational temporal upsampling and denoising framework.

• Temporal Super-Resolution: By jittering sub-pixel positions and rendering at a lower internal resolution, TAA can reconstruct a stable high-resolution output, a technique used in DLSS 2+ and FSR 2.
• Denoising Partner: In real-time ray tracing, TAA's history buffer is crucial for spatiotemporal denoising. It accumulates samples over multiple frames to converge a clean image from a very low sample-per-pixel count.
• Deferred Rendering: Works naturally with G-buffers, as motion vectors and depth are readily available.

TAA is a highly efficient form of anti-aliasing, but its implementation requires careful engineering.

• GPU Cost: Typically adds 0.5-1.5 ms of GPU time per frame. The cost is primarily in the post-processing pixel shader that performs reprojection, blending, and clamping.
• Memory: Requires storage for the history buffer (usually RGBA16F) and often a previous frame's depth/motion vector buffer.
• Pipeline Dependency: Creates a feedback loop, making frames dependent. This can complicate techniques like multi-GPU rendering (Alternate Frame Rendering).
• Temporal Stability: The key metric for TAA quality. A good implementation produces a stable, flicker-free image even with complex camera and object motion.

Reprojection is the geometric process of transforming pixel data (color, depth) from a previous frame's camera viewpoint to align with the current frame's viewpoint. It is the foundational mathematical operation for temporal reuse.

• Core to TAA: TAA uses reprojection to locate where a pixel from the previous frame should be sampled in the current frame, using motion vectors.
• Challenges: Requires accurate per-pixel motion vectors. Errors cause ghosting artifacts where disoccluded content (newly revealed areas) is incorrectly blended.

Motion vectors are per-pixel 2D vectors stored in a render pass. They encode the screen-space displacement of a surface point between two consecutive frames.

• Input for Reprojection: Provide the data needed to map previous-frame pixels to the current frame. Calculated from object transformations and camera motion.
• Precision Requirements: Inaccurate or missing motion vectors (e.g., for complex effects like particle smoke) are a primary source of TAA artifacts like smearing.

A G-buffer is a set of intermediate render targets storing per-pixel geometric and material data from a rasterization pass. It decouples geometry processing from shading.

• Deferred Rendering Link: TAA is often integrated into a deferred rendering pipeline, where it operates on the shaded output, but can use G-buffer data (normals, depth) for disocclusion detection and history rejection.
• Data Sources: Contains world position, surface normal, albedo color, material properties, and motion vectors.

Real-time denoising refers to algorithms that remove stochastic noise from images generated with very few samples per pixel, such as in real-time ray tracing. It shares conceptual ground with TAA.

• Temporal Dimension: Modern real-time denoisers (e.g., SVGF, ReBLUR) are temporally stable, reusing and filtering data across frames much like TAA to accumulate samples.
• Key Difference: Denoisers target high-variance Monte Carlo noise; TAA primarily targets aliasing and shading noise from rasterization.

Neural supersampling (e.g., NVIDIA DLSS, AMD FSR) uses a deep neural network to reconstruct a high-resolution image from a lower-resolution render, often leveraging temporal data.

• Advanced Temporal Accumulation: These systems employ sophisticated temporal accumulation pipelines that supersede or integrate traditional TAA logic, using motion vectors and history buffers.
• Performance vs. Quality: They are a form of temporal super-resolution, trading render resolution for performance while aiming to match or exceed native rendering quality with TAA.

Variable Rate Shading is a GPU hardware feature that allows different regions of an image to be shaded at different frequencies (e.g., 1x1, 2x2 pixel blocks).

• Performance Synergy: VRS can reduce shading cost for peripheral or motion-blurred regions. TAA can help mask the potential coarseness introduced by lower shading rates.
• Integration: The shading rate can be guided by motion vector magnitude, ensuring reduced detail where temporal accumulation will provide sufficient smoothness.