Faithfulness Score

Two complementary metrics that decompose faithfulness into distinct, measurable properties of an explanation.

• Comprehensiveness: Quantifies how much the model's prediction changes when the most important features are removed. A high comprehensiveness score indicates the explanation captured all features the model relied on.
• Sufficiency: Measures if the most important features alone are enough to sustain the original prediction. A low sufficiency score confirms the model needs more than just the highlighted region.
• These metrics are foundational in toolkits like Quantus and provide a standardized benchmark for comparing attribution methods.

In synthetic or semi-synthetic settings, faithfulness can be measured by comparing an explanation to a known, ground-truth signal that was intentionally embedded in the data.

• Synthetic Lesion Injection: A known artifact is digitally inserted into a medical scan; a faithful explanation must exclusively highlight that region.
• Grid Pointing Game: A dataset is constructed where the class label is determined by a specific, known pixel location.
• Rank Correlation: Metrics like Kendall's Tau or Spearman's Rho measure the statistical correlation between the rank order of attribution scores and the true feature importance.

A faithful explanation must be sensitive to meaningful changes in the model or input while remaining robust to irrelevant perturbations.

• Input Invariance: Adding imperceptible noise to the input should not drastically alter the explanation if the model's prediction remains unchanged.
• Model Parameter Randomization: Progressively randomizing a model's weights from the top layer down must cause the explanation to degrade. If the saliency map stays visually similar, it is unfaithful and merely an edge detector.
• Max-Sensitivity: Measures the maximum change in an explanation given a small, constrained input perturbation, bounded by Lipschitz continuity.

Moving beyond correlation to causation, these metrics test if the highlighted features are the cause of the model's decision.

• Causal Faithfulness: Intervenes on the model's internal activations by surgically removing the effect of specific neurons or attention heads identified by the explanation.
• Ablation Gap: Measures the difference in model output when removing features based on the explanation versus removing a random set of equal size. A large gap indicates high faithfulness.
• Necessity vs. Sufficiency: Distinguishes features that are required for the prediction (necessity) from those that can independently trigger it (sufficiency).

A critical distinction in explainable AI: faithfulness is an objective property of the explanation's fidelity to the model's internal mechanics, while plausibility is a subjective measure of how convincing the explanation appears to a human.

• The Interpretability Illusion: A smooth, anatomically plausible saliency map can be completely unfaithful, creating a false sense of security in clinical settings.
• Human-Grounded Evaluation: Plausibility is measured via clinician surveys; faithfulness is measured via perturbation and ablation.
• Regulatory Relevance: Bodies like the FDA prioritize faithfulness over plausibility for SaMD audit trails, as it verifies the model's true reasoning, not just a post-hoc rationalization.

The false sense of security arising from a plausible-looking but unfaithful explanation. A saliency map may appear to highlight a tumor convincingly, yet the model's decision could be driven by a spurious correlation with a scanner artifact. This illusion is a critical safety risk in high-stakes medical diagnosis, where an unfaithful explanation can mislead a clinician into trusting an incorrect prediction.

The property of a model to maintain prediction accuracy under adversarial perturbation. In the context of explainability, an attribution attack can subtly modify an input image so the model's diagnosis remains unchanged, but the generated saliency map is completely altered. A robust model and explanation pair is essential to prevent malicious actors from hiding diagnostic logic.

The process of aligning a clinician's subjective trust in an AI tool with the tool's objective performance. A high faithfulness score is a prerequisite for proper calibration. Without it:

• Over-reliance: Clinician accepts an incorrect diagnosis based on a misleading but confident-looking heatmap.
• Under-reliance: Clinician dismisses a correct diagnosis due to a noisy or counter-intuitive explanation.

An explanation method that seeks to identify the input features that are the actual causes of a model's decision, not just correlated with it. Unlike standard feature attribution, it uses interventions and structural causal models. In medical imaging, this can distinguish whether a model diagnoses pneumonia based on lung opacity (causal) or a chest tube (spurious correlation).

The specific transparency requirements mandated by bodies like the FDA for Software as a Medical Device (SaMD). A documented faithfulness score is becoming a critical part of a SaMD Audit Trail. It provides quantitative evidence that the model's reasoning is stable and can be audited, directly supporting pre-market approval and post-market surveillance.