Captum

Captum implements a comprehensive suite of attribution methods, each satisfying different mathematical axioms:

• Integrated Gradients: Satisfies the completeness axiom by integrating gradients along a path from a baseline to the input
• DeepLIFT: Compares neuron activation to a reference activation, assigning contribution scores based on differences
• GradientShap: Computes SHAP values by integrating gradients along paths defined by a background distribution
• Saliency: A simple gradient-based method showing which input pixels most affect the output
• Input X Gradient: Multiplies the input by its gradient to improve sharpness over raw saliency maps

Beyond input-level explanations, Captum supports attribution with respect to hidden layers and neurons:

• Layer Conductance: Extends Integrated Gradients to measure the importance of individual neurons within a hidden layer
• Internal Influence: Quantifies how much a specific neuron in one layer influences neurons in subsequent layers
• Neuron Conductance: Decomposes a prediction score across the neurons of a given layer, identifying which learned features drive decisions
• Layer Gradient X Activation: Computes coarse localization maps by combining gradients and activations at any convolutional layer, similar to Grad-CAM

Captum includes built-in tools to quantitatively assess explanation quality, addressing the interpretability illusion:

• Faithfulness metrics: Measure correlation between attributed importance and actual model output change when features are perturbed
• Sensitivity analysis: Evaluates explanation robustness to small input perturbations using max-sensitivity and infidelity metrics
• Infidelity metric: Quantifies how well an explanation predicts the model's response to significant perturbations
• Perturbation-based evaluation: Systematically removes or alters features ranked by importance to validate attribution accuracy

Captum supports advanced interpretability for complex, multi-input models common in clinical settings:

• Multi-Modal Attribution: Decomposes predictions across different input modalities (imaging, genomics, clinical text) to identify which data source drives a diagnosis
• Concept Attribution with TCAV: Tests model sensitivity to user-defined clinical concepts (e.g., 'spiculated mass', 'ground-glass opacity') using directional derivatives in activation space
• Feature Ablation: Systematically removes input features or groups to measure their causal impact on predictions
• Token Attribution: For vision-language models, assigns importance to individual text tokens alongside image regions

Captum's Noise Tunnel functionality smooths attribution maps by integrating over multiple noisy copies of the input, reducing visual noise and improving reliability:

• SmoothGrad integration: Averages saliency maps over samples with added Gaussian noise to produce cleaner visualizations
• VarGrad: Computes the variance of gradients under noise to identify regions where the model's reasoning is uncertain
• SmoothGrad for Integrated Gradients: Applies noise smoothing to path-integrated attributions for more robust feature importance
• This is critical for regulatory explainability, where noisy or unstable saliency maps undermine clinical trust

As a first-class PyTorch library, Captum integrates seamlessly with existing medical imaging pipelines:

• Native torch.nn.Module support: Works directly with any PyTorch model without wrapper classes or architectural changes
• Dataloader compatibility: Accepts standard PyTorch DataLoader objects for computing attributions over entire datasets
• GPU acceleration: All attribution algorithms run natively on CUDA-enabled hardware for rapid processing of large 3D medical volumes
• Gradient and perturbation API: Provides a unified interface for both gradient-based and perturbation-based attribution methods
• Visualization utilities: Includes built-in tools for rendering saliency maps, heatmaps, and attribution graphs

An axiomatic attribution method that satisfies the completeness property, ensuring the sum of feature importances equals the difference between the model output at the input and a baseline. It computes attributions by integrating gradients along a straight-line path from a neutral baseline (e.g., black image) to the actual input.

• Key property: Satisfies Sensitivity and Implementation Invariance axioms
• Medical use: Identifies pixel regions driving a diagnostic classification
• Captum integration: captum.attr.IntegratedGradients

Produces coarse localization maps by using gradient signals flowing into the final convolutional layer of a CNN. It highlights discriminative image regions used for classification without requiring architectural changes.

• Output: Class-discriminative heatmap overlaid on the original image
• Medical application: Validating that a pneumonia detector focuses on lung opacities, not text markers
• Captum integration: captum.attr.LayerGradCam

A unified game-theoretic framework that assigns each input feature a Shapley value representing its marginal contribution to the prediction. SHAP values satisfy consistency, local accuracy, and missingness properties.

• Variants: KernelSHAP (model-agnostic), DeepSHAP (deep learning optimized)
• Medical relevance: Quantifies exactly how much each pixel contributed to a malignancy score
• Captum integration: captum.attr.KernelShap, captum.attr.DeepLiftShap

Quantitative metrics that measure how accurately an explanation reflects the model's true reasoning. Evaluated by perturbing attributed features and measuring output change.

• Faithfulness Correlation: Pearson correlation between attribution magnitude and prediction drop upon feature removal
• Monotonicity: Whether removing higher-attributed features consistently causes larger output changes
• Tooling: Captum's captum.metrics module provides faithfulness and sensitivity metrics

A conservation-based decomposition technique that redistributes the prediction score backward through network layers using specific propagation rules (e.g., LRP-ε, LRP-αβ). Unlike gradient methods, LRP does not rely on gradient computation.

• Key advantage: Produces signed relevance maps distinguishing positive from negative evidence
• Medical imaging: Separates regions supporting vs. opposing a tumor classification
• Captum integration: captum.attr.LRP

A Captum utility that applies SmoothGrad by adding Gaussian noise to inputs and averaging attributions across multiple samples. This reduces visual noise in saliency maps and produces more coherent explanations.

• SmoothGrad: Averages gradients over noisy copies of the input
• SmoothGrad Integrated Gradients: Combines noise smoothing with axiomatic attribution
• Medical benefit: Produces anatomically smoother, clinically interpretable heatmaps