SmoothGrad

SmoothGrad operates on the principle that meaningful features are robust to small perturbations, while noise is not. Gaussian noise is added to the input image to create N noisy copies. A standard saliency map (the gradient of the class score with respect to the input) is computed for each copy. The final SmoothGrad map is the pixel-wise average of these N maps. This averaging process cancels out high-frequency, non-robust noise, leaving a cleaner visualization of the true decision boundaries.

Standard sensitivity maps often appear visually noisy and fragmented, making it difficult to discern the actual image structure driving a classification. SmoothGrad directly addresses this by producing maps with sharper, more contiguous regions of high importance. For example, in an image of a dog, a raw saliency map might highlight scattered pixels across the background, while SmoothGrad will produce a more coherent silhouette of the dog itself, clearly separating the subject from irrelevant background features.

The key hyperparameter is the standard deviation (σ) of the added Gaussian noise, defined as a fraction of the input's dynamic range (typically 0-1 for normalized images).

• Low σ (e.g., 0.05): Insufficient noise reduction; the map remains noisy.
• Optimal σ (e.g., 0.1 - 0.2): Effectively cancels noise while preserving sharp feature boundaries.
• High σ (e.g., 0.5+): Over-smooths the map, destroying meaningful detail and creating a blurry, uninformative visualization.

The number of noisy samples (n) controls the quality of the Monte Carlo average. A larger n produces a more stable and less noisy final map, but at a linear increase in computational cost. The law of diminishing returns applies quickly.

• n=1: Equivalent to a standard noisy saliency map.
• n=50: A common default that provides a good balance between quality and speed.
• n=200+: Produces a highly stable map, but the visual difference from n=50 is often negligible for human interpretation.

SmoothGrad is not a standalone attribution method but a meta-technique that can enhance any gradient-based explanation. It can be applied to:

• Standard Saliency Maps: Smoothing the raw input gradients.
• Guided Backpropagation: Sharpening the deconvolutional visualizations.
• Integrated Gradients: Reducing noise in the baseline-interpolated gradient path.
• Grad-CAM: Smoothing the gradient signal used to weight activation maps, potentially leading to more precise localization.

The primary cost of SmoothGrad is a linear increase in computation. Generating a SmoothGrad map requires n forward and backward passes instead of one. For a large model like a Vision Transformer, this can be significant. The trade-off is between computational budget and visual coherence. In practice, this cost is often acceptable for offline analysis and debugging, where a clear, interpretable visualization is more valuable than real-time speed.

An axiomatic method that satisfies the completeness property by accumulating gradients along a straight-line path from a baseline to the input. Unlike SmoothGrad's local noise averaging, Integrated Gradients addresses the saturation problem inherent in standard gradients. The two methods are complementary: SmoothGrad sharpens visualizations, while Integrated Gradients provides theoretically grounded attribution scores.

A visualization technique that modifies standard backpropagation to only propagate positive gradients through ReLU activations, preventing negative contributions from flowing backward. This produces sharper saliency maps than vanilla gradients. SmoothGrad can be applied on top of Guided Backpropagation to further reduce high-frequency noise and produce cleaner feature visualizations.

Quantitative criteria used to evaluate how accurately an attribution map reflects true feature importance. Key protocols include:

• Deletion Metric: Removing important pixels and measuring prediction decay
• Insertion Metric: Adding important pixels to a blurred baseline and measuring probability increase
• ROAR (RemOve And Retrain): Iteratively retraining after removing top-k features These metrics validate whether SmoothGrad's noise reduction preserves attribution fidelity.

A localization technique that uses the gradients of a target class flowing into the final convolutional layer to produce a coarse heatmap. While Grad-CAM provides class-discriminative localization, its resolution is limited by the feature map dimensions. SmoothGrad operates at the pixel level and can complement Grad-CAM by sharpening the fine-grained gradient signals used in the Grad-CAM computation.

An extension of Integrated Gradients that removes the need for a single baseline by averaging attributions over multiple background samples drawn from the training distribution. This shares SmoothGrad's core insight—that averaging over multiple perturbations reduces noise and improves reliability—but applies it to the baseline selection problem rather than input noise injection.