Integrated Gradients

The Completeness Axiom (or Summation to Difference) is the fundamental property that ensures the attribution scores for all input features sum to the difference between the model's output for the input and its output for the baseline. Formally, for input (x) and baseline (x'), the attributions (a_i) satisfy: (\sum_i a_i = F(x) - F(x')). This guarantees the explanation accounts for the entire prediction delta, providing a natural scale for importance scores.

Implementation Invariance ensures that two functionally equivalent models (i.e., models that produce identical outputs for all inputs, regardless of their internal architecture or implementation details) will receive identical feature attributions. This axiom is critical because it means Integrated Gradients explains the function the model computes, not the idiosyncrasies of its implementation. It distinguishes the method from approaches that are sensitive to internal parameterization.

The Sensitivity axiom states that if a model's prediction function does not depend mathematically on a feature, and that feature differs between the input and the baseline, then the attribution for that feature should be zero. Conversely, if the function depends solely on a feature and that feature differs, the attribution should be non-zero. This ensures the method correctly identifies features that are causally relevant to the prediction, avoiding false positives.

The Linearity axiom posits that for a linear combination of two models, the attributions for the combined model are a weighted sum of the individual attributions. Formally, if (F = aF_1 + bF_2), then the attributions for (F) are (a) times the attributions for (F_1) plus (b) times the attributions for (F_2). This property ensures the explanation method behaves predictably and consistently across model ensembles or linearly composed functions.

The choice of baseline is a critical, non-axiomatic parameter that significantly influences the resulting attributions. The baseline represents an input with 'no information' (e.g., a black image, a zero vector, or an average embedding).

• Impact: Attributions explain the prediction relative to the baseline. A poorly chosen baseline can yield uninterpretable results.
• Common Practices: Use a neutral reference (like zero), a distributional average, or a counterfactual input representing an 'absence' of the predicted class.

Integrated Gradients is part of the path methods family, which integrate gradients along a path from baseline to input. The straight-line path (\gamma(\alpha) = x' + \alpha(x - x')) for (\alpha) from 0 to 1 is the simplest and most common.

• Advantages: It is symmetric and satisfies the completeness axiom.
• Alternatives: Other paths (e.g., monotonic) are possible but may violate desirable axioms. The straight-line path provides a unique solution satisfying all core axioms.

The baseline is a critical hyperparameter representing an 'informationless' input. Common choices include:

• Zero vector: A simple all-zero input.
• Mean/Median feature values: Represents an average input.
• Random noise: A random sample from the input distribution.
• Counterfactual baseline: An input representing an opposite class (e.g., a blank image for an object classifier).

The choice significantly impacts attributions. A zero baseline for an image model highlights all non-zero pixels, while a blurred version highlights edges. The baseline should be justified by domain knowledge.

The integral along the straight-line path is approximated numerically. The key parameter is the number of steps (m).

• Trapezoidal rule: The default, summing gradients at interpolated points.
• Left Riemann sum: Uses gradients at the baseline and intermediate points.
• Right Riemann sum: Uses gradients at intermediate points and the actual input.

A common heuristic is to use 20-50 steps. Too few steps cause approximation error; too many increase compute cost with diminishing returns. Convergence should be checked by observing attribution stability as m increases.

The Completeness Axiom (or Summation to Delta) is the core theoretical guarantee: the sum of Integrated Gradients attributions equals the difference between the model's output for the input and the baseline. This serves as a primary implementation sanity check.

• Calculation: sum(attributions_i) ≈ F(input) - F(baseline).
• Deviation: Any significant deviation indicates a bug in the gradient computation or numerical integration.
• Use: This axiom is used to verify the correctness of the implementation before any analysis, ensuring the explanation accounts for the entire prediction delta.

A robust explanation should be stable under small input perturbations. Key validation metrics include:

• Sensitivity-n: Measures the maximum change in attribution when up to n features are perturbed. Lower scores indicate greater robustness.
• Infidelity: Quantifies the error between the explanation's importance scores and the actual change in model output when the input is perturbed according to the explanation. High infidelity suggests the explanation does not faithfully reflect model behavior.
• Implementation: These metrics require generating many perturbed samples (e.g., by adding Gaussian noise) and recomputing predictions and attributions, making them computationally intensive but essential for reliability assessment.

Integrated Gradients is one of several popular attribution methods. Key distinctions:

• vs. SHAP: SHAP is also grounded in game theory (Shapley values) but can be computationally expensive. KernelSHAP is model-agnostic but approximate; DeepSHAP is a faster, model-specific approximation. IG is typically more efficient for differentiable models.
• vs. LIME: LIME fits a local surrogate model (e.g., linear) to explain a prediction. While intuitive, LIME explanations can be unstable and may not faithfully represent the complex model's true decision boundary. IG provides exact attributions for the original model.
• Selection Guide: Use IG for differentiable models (neural networks) where implementation efficiency and theoretical guarantees are priorities. Use SHAP or LIME for non-differentiable models (e.g., tree ensembles).

For image and text models, attributions must be presented intuitively for human analysts.

• Images: Overlay attribution scores as a heatmap (saliency map) on the original image. Use divergent color scales (e.g., red-blue) to show positive/negative contributions.
• Text: Highlight tokens in the input text with color intensity proportional to attribution score.
• Human-AI Agreement: The ultimate test is whether the highlighted features align with domain expert intuition for a set of canonical examples. Low agreement may indicate issues with the baseline choice, model logic, or the need for concept-based methods like TCAV.

Feature attribution is the overarching class of explainability methods to which Integrated Gradients belongs. These methods assign a numerical importance score to each input feature, quantifying its contribution to a specific model prediction.

• Core Goal: To answer the question, "Which parts of this input were most responsible for this output?"
• Output Format: Typically a vector of scores (an attribution map) with the same dimensionality as the input.
• Applications: Used for debugging model logic, establishing trust, and meeting regulatory requirements for automated decision systems.

SHAP is a unified framework for model interpretation based on cooperative game theory's Shapley values. Like Integrated Gradients, it provides a theoretically grounded method for feature attribution.

• Theoretical Basis: Attributes prediction by calculating the average marginal contribution of a feature across all possible coalitions (subsets) of other features.
• Key Property: Uniquely satisfies the axioms of local accuracy, missingness, and consistency.
• Contrast with IG: While IG uses a specific path (baseline to input), SHAP considers all possible paths, making it computationally more expensive but offering a different axiomatic guarantee.

A saliency map is a visual explanation technique, most commonly applied to image models, that highlights the regions of an input most influential for a prediction. Integrated Gradients can be used to generate saliency maps.

• Visual Output: Creates a heatmap overlay on an image, where intensity indicates feature importance.
• Implementation: For image inputs, the attribution scores from IG are aggregated per-pixel or per-region to form the visual map.
• Use Case: Critical in medical imaging and autonomous driving to verify a model is focusing on clinically or contextually relevant features (e.g., a tumor, a stop sign) rather than spurious background correlations.

The baseline input is a fundamental hyperparameter in the Integrated Gradients method. It represents a neutral starting point with "no information" from which importance is accumulated.

• Definition: The input from which the integration path begins (α=0). The choice of baseline significantly impacts the resulting attributions.
• Common Choices: For images, a black or blurred image. For text, a padding token or zero embedding. For tabular data, a vector of feature means or zeros.
• Interpretation: The attribution scores explain the prediction relative to this baseline. A good baseline should represent an absence of the signal being detected.

Perturbation analysis is a general validation technique for explanations. It tests the causal relationship between features identified as important and the model's output by systematically modifying them.

• Method: Features ranked highly by an attribution method (like IG) are removed or altered (e.g., masked, set to baseline), and the change in the model's prediction is observed.
• Validation Use: A faithful explanation should cause a large prediction drop when its top features are perturbed. This is the basis for metrics like Faithfulness Score and Sufficiency.
• Direct Application: Used to empirically validate the attributions produced by Integrated Gradients.

The Completeness Axiom (also called Summation to Delta) is the core mathematical property that defines Integrated Gradients. It states that the attributions for all input features must sum to the difference between the model's output for the input and the baseline.

• Formula: Σ (Attribution_i) = F(input) - F(baseline), where F is the model.
• Implication: The explanation fully accounts for the model's prediction change. No importance is missing or unaccounted for.
• Significance: This axiom provides an intuitive sanity check and is a key reason for IG's adoption, ensuring the attribution is a complete decomposition of the prediction difference.