Integrated Gradients

The Sensitivity axiom states that if an input differs from a baseline in exactly one feature, and that difference causes a change in the model's prediction, the differing feature must receive a non-zero attribution. A corollary is Completeness, which mandates that the sum of all feature attributions must equal the difference between the model's output for the input and the baseline. This ensures no attribution is created or destroyed, providing a full accounting of the prediction.

Two functionally equivalent networks—meaning they produce identical outputs for all possible inputs—must yield identical attributions, regardless of their internal architecture. This property ensures that Integrated Gradients explains the mathematical function a network has learned, not the incidental specifics of its implementation. A method that relies on specific layer activations, like raw gradients, would violate this axiom.

The attribution for a model that is a linear combination of two sub-models must be the same linear combination of the attributions from each sub-model. This property preserves the structure of ensemble models. For example, if a final prediction is an average of two networks, the Integrated Gradients attribution is simply the average of the attributions computed for each network individually.

Two input variables that play the exact same functional role in the network and have identical values must receive identical attributions. This ensures fairness in the explanation process. If a model treats two features symmetrically, the explanation method must not arbitrarily assign more importance to one over the other. This is a fundamental requirement for a coherent explanation.

The attribution for a feature is computed by accumulating its gradients along a straight-line path from a baseline input (representing an absence of signal, like a black image) to the actual input. Mathematically, this is the path integral of the model's gradient with respect to that feature. This process captures how the model's sensitivity to the feature evolves as information is progressively introduced.

The choice of baseline is a critical hyperparameter that encodes the definition of 'absence' for the input domain. A common baseline for image networks is a black image (all zeros) , while for text models it might be a sequence of zero-embedding tokens. The explanation answers the question: 'Why did the model predict this output for the input, relative to this specific baseline?' A poorly chosen baseline can lead to misleading attributions.

Integrated Gradients satisfies the completeness axiom, meaning the sum of all feature attributions exactly equals the difference between the model's output for the input and the baseline. This property is critical for algorithmic auditing under frameworks like the EU AI Act, where every unit of a decision must be accounted for. Auditors use this to verify that protected attributes like race or gender contribute zero to credit or hiring decisions.

• Provides a mathematically complete decomposition of a prediction
• Enables verification of fairness constraints in high-risk AI systems
• Used in conformity assessments to demonstrate non-discrimination

Radiologists use Integrated Gradients to validate that a convolutional neural network is focusing on clinically relevant regions in an X-ray or MRI, rather than spurious correlations like hospital markers or scanner artifacts. By setting a black image baseline, the method highlights the exact pixels that contributed to a positive diagnosis.

• Validates that models attend to pathological structures, not metadata
• Supports human-in-the-loop review for life-critical decisions
• Integrated into FDA-cleared computer-aided detection software

For text classification and sentiment analysis, Integrated Gradients computes token-level importance scores by interpolating between a zero-embedding baseline and the actual word embeddings. This reveals which words most influenced a prediction, helping engineers debug hallucination patterns and toxic content classifiers.

• Identifies spurious lexical shortcuts the model has learned
• Explains why a review was classified as negative based on specific adjectives
• Used to audit hate speech detection models for bias

Fraud analysts use Integrated Gradients to understand why a transaction was flagged as suspicious. By setting a baseline representing a 'normal' transaction profile, the method attributes the anomaly score to specific features like transaction amount, geographic location, or time of day. This provides an auditable trail for Suspicious Activity Reports filed with regulators.

• Generates feature-level evidence for compliance filings
• Reduces false positive investigation time by surfacing key drivers
• Integrates with automated decision logging systems for immutability

The choice of baseline is the most critical design decision when applying Integrated Gradients. A black image works for object recognition, a zero embedding vector for text, and a blurred or random input for domain-agnostic scenarios. Poor baseline selection can lead to noisy attributions or explanations that ignore important features. Multiple baselines should be tested and averaged for robustness.

• Black baseline: Standard for computer vision tasks
• Zero baseline: Common for tabular and text data
• Gaussian noise baseline: Useful when no natural 'absence' exists
• Averaging over multiple baselines improves explanation fidelity

Unlike raw saliency maps which only show local sensitivity, Integrated Gradients satisfies the sensitivity axiom—if a feature changes and the prediction changes, that feature receives non-zero attribution. This avoids the saturation problem where gradients become near-zero for features that strongly influence the output. Compared to DeepLIFT, Integrated Gradients requires no custom backpropagation rules and is implementation-invariant.

• Saliency Maps: Fast but violate sensitivity; miss saturated features
• Grad-CAM: Good for localization but limited to CNNs
• DeepLIFT: Requires reference-dependent rules per layer
• Integrated Gradients: Axiomatically sound and model-agnostic