Integrated Gradients

The attributions for all input features must sum to the difference between the model's output for the input and its output for the baseline. This ensures no importance is lost or created during the attribution process.

• Mathematical Form: Σᵢ IGᵢ(x) = F(x) - F(x')
• Practical Impact: Guarantees a full accounting of the prediction, making it suitable for regulatory audits where a complete explanation is required.
• Contrast: Simpler methods like input * gradient do not satisfy completeness, leading to potentially misleading partial explanations.

If an input feature and a baseline differ in exactly one feature, and changing that feature alters the model's prediction, that feature must receive a non-zero attribution. If the model is mathematically flat with respect to a feature, it receives zero attribution.

• Core Idea: The method is sensitive to features that actually matter for a specific prediction.
• Violation Example: A method that relies on local gradients can assign zero importance to a feature even when it is critical, if the gradient at the specific input point is flat.

Two functionally equivalent neural networks—meaning they always produce the same output for every possible input—must receive identical attributions, regardless of their internal architecture or training parameters.

• Key Distinction: This separates Integrated Gradients from methods that rely on inspecting internal model weights or activations, such as attention weights.
• Practical Benefit: The explanation is a property of the model's mathematical function, not its specific implementation, ensuring consistency across different versions of a model that learn the same function.

If a model is a linear combination of two sub-models, the attribution for the combined model is the same linear combination of the attributions from the sub-models.

• Formal Definition: IGᵢ(aF₁ + bF₂) = aIGᵢ(F₁) + bIGᵢ(F₂)
• Ensemble Models: This property is crucial for explaining ensemble models or models with residual connections, ensuring attributions decompose logically.
• Uniqueness: Integrated Gradients is the unique path method that satisfies this axiom alongside the others.

If a model's output is symmetric with respect to two input features—meaning swapping their values does not change the prediction—then those two features must receive identical attributions when their input and baseline values are also swapped.

• Fairness Application: This ensures that functionally identical features are treated equally by the explanation, which is a foundational requirement for algorithmic fairness audits.
• Example: In a model where two biomarkers are perfect proxies for each other, they must receive equal importance scores.

The attribution is computed by accumulating the gradients of the model's output along a straight-line path from a non-informative baseline to the actual input. This path integral captures how the model's sensitivity to each feature evolves.

• Baseline Choice: The selection of the baseline is critical. A common choice is a zero-embedding or a blurred image, representing an absence of information.
• Expected Gradients: An extension that averages over multiple baselines to reduce noise and provide a more robust explanation.

Integrated Gradients can highlight the specific pixels in a chest X-ray that most influenced a model's finding of pneumonia. By integrating gradients from a black baseline image to the patient's scan, the method generates a saliency map that localizes the pathology. This allows a radiologist to verify that the model is focusing on the lung parenchyma and not on spurious correlations like laterality markers or chest tubes, building trust in the automated diagnosis.

In digital pathology, a model may classify a whole-slide image as malignant. Integrated Gradients decomposes this prediction to identify the specific clusters of cells with atypical nuclei that drove the decision. The pathologist can then review these high-attribution regions to confirm mitotic figures or architectural distortion, effectively using the explanation as a triage tool that reduces the search space on a gigapixel image.

When a deep learning model predicts a pathogenic variant from a patient's genomic sequence, Integrated Gradients assigns importance scores to each nucleotide. This reveals the precise missense mutations or splice-site disruptions that contributed to the classification. For a clinical geneticist, this attribution provides a ranked list of candidate causal variants, accelerating the interpretation of complex whole-exome sequencing data.

An ICU model fusing vital signs, lab values, and clinical notes can predict sepsis onset hours in advance. Integrated Gradients attributes the risk score back to each input feature, showing that a rising lactate level and a drop in platelet count were the primary drivers, while a specific phrase in a nursing note provided secondary confirmation. This multimodal explanation gives the care team a transparent, data-driven rationale for early intervention.

For a model detecting diabetic retinopathy in fundus photographs, Integrated Gradients generates a pixel-level heatmap overlaying the retinal image. The explanation highlights microaneurysms, hemorrhages, and exudates that the model used for its grading. This visual evidence allows an ophthalmologist to quickly validate the model's severity assessment against established clinical grading scales like the ETDRS protocol, streamlining the screening workflow.

A 1D convolutional neural network classifies a 12-lead ECG as showing atrial fibrillation. Integrated Gradients computes the contribution of each time step in each lead, revealing that the model focused on the absence of P-waves and an irregularly irregular R-R interval in lead II. This explanation maps directly to the diagnostic criteria a cardiologist uses, demonstrating that the model has learned clinically relevant waveform morphology.