Stability Score

A Stability Score is a quantitative metric that measures the consistency of explanations generated for similar inputs or under small, semantically-preserving perturbations. Its core purpose is to assess the robustness of an explanation method itself, ensuring that the attributed feature importance does not change erratically for minor, inconsequential changes to the input. A high score indicates that the explanation method produces reliable and trustworthy attributions, which is critical for user trust and regulatory compliance.

Stability is primarily measured by applying controlled perturbations to an input and observing the variation in the resulting explanations. Common perturbation strategies include:

• Additive Noise: Adding small amounts of Gaussian noise to numerical features or embeddings.
• Feature Masking: Randomly masking a small percentage of non-critical input tokens or pixels.
• Synonym Replacement: Swapping words with their synonyms in text inputs. The score is often calculated as the inverse of the explanation variance or the cosine similarity between the original explanation vector and the explanations for perturbed inputs. A method with low variance or high average similarity scores highly.

Stability is intrinsically linked to, but distinct from, Faithfulness and Infidelity metrics.

• Faithfulness measures how accurately an explanation reflects the model's true reasoning for a single input.
• Infidelity quantifies how much the explanation fails to predict model output changes when the input is perturbed according to the explanation's own importance scores.
• Stability assesses the consistency of explanations across multiple similar inputs, regardless of their ground-truth faithfulness. An explanation can be stable but unfaithful (consistently wrong) or faithful but unstable (correct but fragile).

The Stability Score evaluates the explanation method (e.g., SHAP, LIME, Integrated Gradients), not the underlying model. Therefore, it is a model-agnostic metric. A stability test must be conducted for each combination of explanation technique and model architecture. For instance, gradient-based methods like Integrated Gradients may demonstrate higher stability for smooth, differentiable models, while perturbation-based methods like LIME might show more variance. This characteristic makes stability a key criterion for selecting an explanation framework for a production system.

In production AI systems, unstable explanations are a major operational risk. They can lead to:

• Eroded User Trust: Inconsistent rationales for similar user queries confuse stakeholders and undermine confidence.
• Flawed Audits: Regulatory or internal audits relying on explanations cannot draw reliable conclusions if the attributions are non-deterministic.
• Unreactive Monitoring: Drift detection systems that monitor explanation distributions will trigger false alerts due to methodological instability rather than actual model or data drift. Engineering teams therefore prioritize explanation methods with provably high stability scores for deployment.

A fundamental sanity check for stability is the Model Randomization Test. This test evaluates if an explanation method is sensitive to the model's actual learned patterns. The procedure is:

1. Calculate explanations using the fully trained model.
2. Progressively randomize the model's layers (starting from the top).
3. Re-calculate explanations for the randomized model. A robust explanation method should produce significantly different (less stable) results for the randomized model compared to the trained one. If the explanations remain stable even after model randomization, the method may be insensitive to model parameters and thus not truly explaining the model's behavior.

The primary application of a Stability Score is to evaluate the intrinsic robustness of an explanation method (e.g., SHAP, LIME, Integrated Gradients). A high score indicates the method produces consistent attributions for similar data points, confirming it is not overly sensitive to irrelevant noise. This is a prerequisite for trusting any post-hoc explanation in production.

• Core Function: Acts as a sanity check for explanation techniques.
• Example: If SHAP values for a loan applicant's 'income' feature fluctuate wildly for applicants with identical profiles, the method's low stability score flags it as unreliable for audit purposes.

In regulated industries (finance, healthcare), explanations for automated decisions must be stable and reproducible. A Stability Score provides quantitative evidence that an AI system's rationale is consistent, supporting compliance with regulations like the EU AI Act or right to explanation mandates.

• Use Case: Demonstrating to auditors that a credit denial explanation is not an artifact of a fragile interpretation method.
• Process: Generate explanations for a validation set of similar cases; a high aggregate stability score provides empirical support for the explanation's trustworthiness.

Engineers use stability analysis to diagnose model flaws. Unstable explanations for a class of inputs can reveal that the model's decision boundary is overly complex or that it relies on spurious correlations that are not robust to slight variations.

• Diagnostic Signal: Low stability often correlates with areas where the model has poor generalization.
• Actionable Insight: Guides data collection or regularization efforts to smooth the model's response in unstable regions, leading to more robust and reliable predictions.

When multiple explanation techniques are available (e.g., SHAP vs. LIME vs. Saliency Maps), the Stability Score serves as a key comparative metric. Data scientists can benchmark methods on a held-out consistency dataset to select the most robust one for their specific model and data domain.

• Benchmarking Framework: Part of a comprehensive explainability evaluation suite alongside Faithfulness and Completeness scores.
• Outcome: Enables the selection of an explanation method that provides not just plausible, but consistently reliable insights.

For a human expert to trust and simulate a model's reasoning, the provided explanations must be predictable. Erratic explanations for minor input changes undermine trust. A documented high Stability Score assures users that the explanations reflect a coherent underlying logic they can learn and rely upon.

• Human-in-the-Loop: Stable explanations improve the Human-AI agreement metric, as experts can form a consistent mental model of the AI's behavior.
• Result: Facilitates smoother human-AI collaboration in critical domains like medical diagnosis or financial analysis.

Explanation robustness is a defense against adversarial attacks targeting interpretability itself. An attacker might seek to generate honeypot explanations that hide a model's true reasoning. Monitoring the Stability Score under adversarial perturbations can expose such vulnerabilities.

• Security Application: Part of preemptive algorithmic cybersecurity for AI systems.
• Method: Apply small, adversarial perturbations to inputs and measure the resulting change in explanations. A significant drop in stability indicates the explanation method is not locally faithful and can be manipulated.

A faithfulness score is a quantitative metric that measures how accurately an explanation reflects the true reasoning process or causal factors of the underlying model for a given prediction. It is a core companion metric to stability.

• Measures Causal Alignment: Evaluates if the features highlighted by the explanation are the ones the model actually used, not just correlated.
• Common Evaluation Methods: Often measured via perturbation analysis, where features deemed important are removed or altered to see if the prediction changes as expected.
• Contrast with Stability: While stability measures consistency across similar inputs, faithfulness measures correctness for a single input. An explanation can be stable but unfaithful if it consistently highlights the wrong features.

Explanation robustness is the property of an explanation method to produce consistent and stable attributions for a given prediction when the input or model is subjected to minor, semantically-preserving perturbations. A stability score is a direct, quantitative measure of this property.

• Defines the Goal: Robustness is the desired characteristic; stability is its measurable outcome.
• Perturbation Types: Includes adding noise, paraphrasing text, or applying small image transformations that should not change the semantic meaning or a human's explanation.
• Failure Modes: Non-robust explanations can be adversarially manipulated, where tiny, imperceptible input changes cause wildly different feature attributions, undermining trust.

Sensitivity analysis in explainability evaluates how small changes in the input features affect both the model's prediction and the generated explanation. It is the methodological foundation for calculating a stability score.

• Dual Output Analysis: Tracks the delta in the model's prediction score and the delta in the explanation's feature importance scores.
• Quantifies Local Smoothness: Measures whether the explanation function is locally Lipschitz continuous around the input.
• Operationalizes Stability: A common stability score is the average sensitivity across many small perturbations, where lower sensitivity indicates higher stability.

Infidelity is an explanation metric that quantifies the degree to which an explanation fails to accurately reflect the model's output when the input is perturbed according to the explanation's own importance scores. It is an instability measure.

• Perturbation via Explanation: The input is modified by adding noise or masking features weighted by the explanation's importance scores. A faithful, stable explanation should predict the resulting change in model output.
• Mathematical Definition: (\text{Infid}(\phi, f, x) = \mathbb{E}_{I\sim \mu_I}[(I^T \phi(f, x) - (f(x) - f(x - I)))^2]), where (\phi) is the explanation, (f) is the model, (x) is the input, and (I) is a perturbation.
• High Infidelity = Low Stability: A high infidelity score indicates the explanation's attributed importance does not match the model's actual local behavior, signaling instability.

Perturbation analysis is an explanation validation technique that systematically modifies or removes input features to observe the resulting changes in the model's output. It is the primary experimental technique for assessing stability and faithfulness.

• Core Validation Mechanism: Used to compute stability scores, faithfulness scores, and infidelity.
• Perturbation Strategies:
  • Feature Ablation: Setting features to zero or a baseline value.
  • Feature Noise: Adding Gaussian noise.
  • Adversarial Perturbations: Applying worst-case small perturbations.
• Stability Calculation: The variance or average distance between explanations generated for the original and perturbed inputs is a direct stability metric.

Local fidelity is a property of a post-hoc explanation that measures how well the explanation approximates the behavior of the complex model in the immediate vicinity of a specific input instance. Stability is a necessary condition for high local fidelity.

• Local Surrogate Models: Methods like LIME explicitly create a simple, interpretable model (e.g., linear regression) that approximates the complex model locally. The fidelity of this surrogate is its local fidelity.
• Requires Consistency: For the surrogate's explanation to be credible, the complex model's behavior must be relatively stable and linear in that local region. High input-output instability makes creating a high-fidelity local surrogate impossible.
• Evaluation: Measured by the accuracy of the surrogate model in predicting the complex model's outputs for perturbed samples near the instance of interest.