Implementation Invariance

The principle that the sum of all feature attributions must equal the difference between the model's output for the target input and a baseline reference output. Also known as the efficiency axiom in cooperative game theory, this constraint ensures that the explanation fully accounts for the model's prediction. Methods like Integrated Gradients and DeepLIFT explicitly satisfy completeness, while simpler gradient-based approaches often violate it. This axiom is critical for auditing high-stakes decisions where every unit of the output must be accounted for.

A fundamental requirement stating that if a model's output is mathematically independent of a feature, that feature must receive an attribution of exactly zero. This prevents the explanation from assigning spurious importance to irrelevant inputs. The axiom extends to the concept of dummy features in game theory: a player who contributes nothing to any coalition must receive zero payout. Violations of sensitivity-n indicate that an attribution method is leaking importance to non-causal features, undermining trust in the explanation.

A formal evaluation framework that judges feature attribution methods by their adherence to a set of mathematical axioms, including implementation invariance, completeness, sensitivity-n, and linearity. Proposed by Sundararajan et al. (2017), this framework demonstrated that Integrated Gradients is the unique path method satisfying key axioms. The framework provides a rigorous alternative to qualitative human evaluation of saliency maps, enabling objective comparison of explanation techniques.

A class of attribution techniques that define feature importance by integrating gradients along a continuous path in the input space from a baseline to the target input. Integrated Gradients uses a straight-line path, but alternative paths can encode different notions of importance. Path methods inherently satisfy the implementation invariance and completeness axioms when the path function is smooth. The choice of baseline—often a zero vector or blurred input—significantly influences the resulting attribution.

A quantitative metric that evaluates the faithfulness of an attribution method by measuring the error between the model's actual output change and the dot product of the attribution vector with a significant input perturbation. Formally, it quantifies how well the explanation predicts the model's response to meaningful interventions. A low infidelity score indicates that the attribution reliably captures the model's local behavior. This metric is particularly useful for detecting when explanations violate implementation invariance in practice.

A robustness metric for attributions that measures the maximum change in the explanation under small, adversarial input perturbations. It quantifies the stability of a saliency map by computing the Lipschitz constant of the attribution function locally. Explanations with high local Lipschitz estimates are brittle and can change dramatically with imperceptible input noise, a phenomenon related to shattered gradients. This metric is essential for ensuring that implementation-invariant attributions are also practically stable.