Saliency Map

The most common saliency maps are computed by taking the gradient of the target class score with respect to the input image pixels. This calculation reveals how much a tiny change in each pixel's intensity would affect the model's confidence in its prediction.

• Vanilla Gradient: The simplest form, computing the raw gradient via backpropagation to the input layer.
• Guided Backpropagation: Modifies the backward pass to only propagate positive gradients through ReLU activations, suppressing negative contributions and producing sharper, less noisy visualizations.
• SmoothGrad: Reduces visual noise by averaging gradients over multiple copies of the input perturbed with small amounts of Gaussian noise.

Saliency maps are inherently class-specific, highlighting only the evidence for the predicted class rather than all salient objects in an image. This property is critical for debugging model behavior.

• A saliency map for a 'cat' prediction will highlight feline features like ears and whiskers, even if a dog is also present in the background.
• This discriminative nature distinguishes saliency maps from generic edge detectors or attention maps that may respond to any high-contrast region.
• Comparing saliency maps for different classes on the same image reveals how the model shifts its focus to make alternative classifications.

Standard saliency maps inherit the spatial resolution limitations of the model architecture. In classification networks, the final convolutional feature maps are significantly downsampled from the input resolution.

• A 224x224 input image may produce a 7x7 saliency map at the final convolutional layer, requiring upsampling to overlay on the original image.
• This coarse resolution can obscure fine-grained details, making it difficult to pinpoint small lesions in medical images.
• Techniques like Score-CAM and FullGrad attempt to mitigate this by aggregating attribution signals from multiple layers to produce higher-fidelity maps.

Research has demonstrated that some saliency methods produce visually similar heatmaps even when the model's weights are randomized or the data labels are permuted. This reveals a critical failure of faithfulness.

• Edge detectors and frequency priors in the input can dominate the visualization, making the map appear plausible without reflecting the model's actual reasoning.
• A faithful saliency method must pass a model parameter randomization test: the explanation should change substantially when the model's learned parameters are destroyed.
• This insight has driven the development of more robust methods like Integrated Gradients, which satisfies the completeness axiom and is less susceptible to this illusion.

In medical imaging, saliency maps are not just debugging tools—they are part of the clinician-in-the-loop validation process. Radiologists compare heatmaps against anatomical ground truth to calibrate trust.

• A saliency map for a pneumonia classifier should highlight lung opacities, not the laterality marker text burned into the X-ray.
• Domain-specific saliency constrains heatmaps using anatomical atlases to penalize activations outside physiologically plausible regions.
• This verification step is essential for building a SaMD audit trail, documenting that the model's focus aligns with clinically relevant pathology before a diagnosis is accepted.

Saliency maps themselves can be manipulated through adversarial perturbations designed to produce misleading explanations while preserving the original classification.

• An attacker can modify an image so that a correctly classified malignant tumor produces a saliency map highlighting healthy tissue instead of the lesion.
• This undermines trust calibration, as a clinician reviewing the manipulated explanation would reasonably doubt the model's reliability.
• Defending against attribution attacks requires explanation regularization during training and monitoring the faithfulness score of explanations in production to detect anomalies.