Feature Attribution

Gradient-weighted Class Activation Mapping produces coarse localization heatmaps by using the gradient of a target class flowing into the final convolutional layer. It highlights the image regions most influential for a prediction without requiring architectural changes.

• Computationally efficient—requires a single backward pass
• Produces class-discriminative visualizations
• Widely validated for chest X-ray and retinal fundus classification
• Does not require gradient integration or multiple forward passes

An axiomatic attribution method that satisfies the completeness property: the sum of feature attributions equals the difference between the model output at the input and at a baseline. It integrates gradients along a straight-line path from a baseline (e.g., black image) to the actual input.

• Satisfies Sensitivity and Implementation Invariance axioms
• Baseline choice critically affects explanation quality
• Computationally intensive—requires 50–200 interpolation steps
• Commonly used in FDA submission documentation for SaMD

SHapley Additive exPlanations unifies six explanation methods under Shapley values from cooperative game theory. Each feature receives an importance value representing its marginal contribution to the prediction, averaged over all possible feature coalitions.

• Provides both local (per-prediction) and global explanations
• Computationally expensive for high-dimensional images—DeepSHAP approximates values for deep networks
• KernelSHAP variant works with any black-box model
• Used in pathology to verify that nuclei morphology drives grading decisions

LRP redistributes the prediction score backwards through the network using conservative propagation rules. Unlike gradient-based methods, LRP does not rely on gradient computation and produces pixel-level relevance heatmaps with high spatial fidelity.

• Alpha-Beta rule controls positive vs. negative relevance flow
• Particularly effective for deep networks with ReLU activations
• Produces sharper, less noisy heatmaps than simple gradient methods
• Validated for brain tumor segmentation in MRI and CT pulmonary nodule analysis

Local Interpretable Model-agnostic Explanations approximates any classifier locally with an interpretable surrogate model (e.g., sparse linear model). It perturbs the input by segmenting it into superpixels and observes how predictions change.

• Fully model-agnostic—works with any black-box classifier
• Explains individual predictions, not global model behavior
• Superpixel segmentation quality directly impacts explanation coherence
• Useful for validating that a dermatology classifier focuses on lesion boundaries, not background skin

Randomized Input Sampling for Explanation is a black-box method that generates saliency maps by probing the model with thousands of randomly masked versions of the input. It averages the model's outputs, weighted by each random mask, to produce a probability-weighted importance map.

• Requires no gradient access—works with any model API
• Mask resolution trades off between detail and computational cost
• Produces smoother saliency maps than gradient-based methods
• Effective for validating third-party diagnostic APIs where internal weights are inaccessible