Sensitivity-n

Sensitivity-n states that if a feature x_i does not mathematically influence the model's output F(x)—meaning the partial derivative of F with respect to x_i is zero everywhere in the local input domain—then the attribution assigned to x_i must be exactly zero. This prevents attribution methods from hallucinating importance for features the model ignores.

• Formal Definition: If ∂F(x)/∂x_i = 0 for all points in a neighborhood, then Attribution(x_i) = 0.
• Contrast with Sensitivity: Sensitivity (the 'n' is often dropped) requires non-zero attribution when a feature does affect the output; Sensitivity-n handles the opposite case.
• Violation Example: A method that assigns non-zero importance to a constant input pixel that is multiplied by a zero weight fails this axiom.

While Completeness ensures the sum of attributions equals the output difference, Sensitivity-n ensures individual feature fidelity. A method can satisfy Completeness by distributing attribution across all features, but it fails Sensitivity-n if it assigns a non-zero score to a feature with a zero gradient.

• Completeness: Σ Attributions = F(x) - F(baseline)
• Sensitivity-n: If feature is functionally irrelevant, its attribution must be 0.
• Combined Strength: Together, these axioms guarantee that the attribution vector is both exhaustive (sums correctly) and sparse (ignores truly unused features).

Sensitivity-n is one of the key axioms proposed by Sundararajan, Taly, and Yan in their 2017 paper on Integrated Gradients. It serves as a critical filter for evaluating the mathematical soundness of any attribution technique.

• Benchmark Axiom: Methods like DeepLIFT and LRP were designed with this constraint in mind.
• Failure Case: Raw Gradient × Input can violate Sensitivity-n when the gradient is zero but the input value is non-zero, incorrectly yielding a zero attribution for a relevant feature, or vice versa in edge cases.
• Guarantee: Integrated Gradients provably satisfies Sensitivity-n by integrating gradients along a path, ensuring that features with zero gradient everywhere contribute nothing.

When debugging a model, Sensitivity-n guarantees that features shown with zero attribution are genuinely disconnected from the prediction. This allows engineers to confidently prune irrelevant inputs or identify dead neurons.

• Feature Selection: Safely remove features that consistently receive zero attribution across a dataset.
• Architecture Validation: If a known critical feature receives zero attribution, it signals a vanishing gradient problem or a disconnected sub-network.
• Compliance: For regulated industries, proving that a protected attribute (e.g., race) has zero attribution requires an axiom like Sensitivity-n to ensure the explanation is not hiding its influence.

Sensitivity-n works in tandem with Implementation Invariance. Two functionally identical networks must not only produce the same output but also the same attributions. Sensitivity-n ensures that if one network architecture ignores a feature, the attribution method cannot arbitrarily assign importance to it just because of a different internal weight configuration.

• Functional Equivalence: If Network A and Network B compute the exact same function F(x), their attributions for a feature with zero gradient must both be zero.
• Prevents Gaming: Without this axiom, an attribution method could be manipulated by simply adding a disconnected computational branch to the network.

Sensitivity-n only applies to features with a strictly zero gradient in a local neighborhood. It does not require zero attribution for features with a small but non-zero influence, which is governed by the related Sensitivity axiom.

• Local vs. Global: A feature might have a zero gradient at a specific input point but influence the output elsewhere. Sensitivity-n applies locally.
• Discrete Inputs: For text or categorical data, gradients are not directly defined, requiring embedding-space approximations.
• Shattered Gradients: In deep networks, noisy gradients can make it computationally difficult to verify if a gradient is truly zero or just numerically small, complicating strict enforcement.