3D Saliency Map

The core mechanism involves backpropagating the target diagnostic prediction score through the 3D convolutional or transformer network to compute a gradient for every voxel in the input volume. The magnitude of this gradient indicates the voxel's influence on the prediction. Techniques like Guided Backpropagation or Integrated Gradients are adapted to 3D to prevent gradient saturation and ensure smooth, coherent attributions across contiguous anatomical structures.

A critical quality metric is whether the highlighted voxels correspond to known anatomical or pathological structures. Effective 3D saliency maps exhibit spatial coherence, forming contiguous blobs rather than scattered noise. Regularization techniques, such as total variation denoising or penalizing high-frequency spatial components, are often applied during map generation to enforce smoothness and align the explanation with physiological tissue boundaries.

Interpreting a 3D heatmap requires specialized visualization. The saliency volume is typically rendered as a color overlay on the original scan and viewed in standard radiological planes: axial, coronal, and sagittal. Maximum Intensity Projection (MIP) is a common technique used to flatten the 3D saliency into a 2D view, showing only the voxels with the highest attribution scores along a projection line, which helps radiologists quickly identify the most critical regions.

For dynamic imaging modalities like functional MRI (fMRI) or contrast-enhanced CT, the saliency map gains a temporal dimension. A 4D Saliency Map highlights not just where but when specific voxels are most influential. This is crucial for understanding model decisions based on perfusion dynamics or neural activation patterns over time, ensuring the explanation captures the full spatiotemporal context of the physiological process.

The faithfulness of a 3D saliency map is validated through 3D perturbation studies. This involves systematically occluding or replacing the most salient 3D regions (e.g., a segmented nodule) with baseline values and measuring the resulting drop in the model's diagnostic confidence. A faithful map will show a sharp, monotonic decline in performance as the highly attributed voxels are removed, confirming that the model's reasoning is anchored to the correct pathology.

To move from a continuous heatmap to a discrete, quantifiable explanation, 3D saliency maps are often fused with semantic segmentation masks. By averaging the saliency scores within a segmented organ or lesion boundary, one can generate a single region-level attribution score. This bridges the gap between voxel-level explanations and clinically meaningful metrics like tumor burden or organ volume, making the AI's reasoning auditable against established radiological criteria.