Faithful Rationales

The explanation must reflect the true causal drivers of the prediction, not correlated artifacts. A faithful rationale changes if and only if the model's internal decision boundary changes. This is tested via intervention studies: ablating the features cited as important must degrade the model's confidence in the predicted class proportionally. If removing a highly-weighted token does not flip the output, the rationale is merely plausible.

A faithful rationale identifies the minimal sufficient input subset required for the prediction. It must balance:

• Completeness: Capturing all features necessary to reach the decision threshold.
• Sufficiency: The identified subset alone must be enough to trigger the same output with high confidence. A rationale that omits a critical token is incomplete; one that includes irrelevant tokens is not minimal.

A human observer, given the rationale and the input, must be able to correctly anticipate the model's output on a held-out example. This forward-simulation test measures practical faithfulness. If the explanation highlights 'positive sentiment' but the model consistently misclassifies sarcasm, the rationale fails the simulatability criterion because it does not encode the model's true vulnerability.

Faithful rationales must be robust against the 'bypass' phenomenon, where an explainer model learns to ignore the input and generate generic, high-confidence justifications. A faithful system enforces input-rationale coupling: the generated text must have zero mutual information with the output class when the input features are masked. This prevents the rationale generator from collapsing into a class-conditional language model.

The rationale must be derived from the same latent representations used for prediction. In self-explaining architectures, this is enforced by design: the explanation is a direct transformation of the final hidden state. In post-hoc systems, faithfulness requires that the explainer has access to the model's attention weights, gradient flows, or activation maps—not just the output logits. Explanations generated from logits alone are inherently susceptible to rationalization.

A faithful rationale remains logically consistent when the input is semantically preserved but syntactically varied. If paraphrasing a sentence changes which tokens the rationale highlights as important, the explanation is brittle and unfaithful. This is measured via input invariance tests: applying synonym substitution or back-translation should not alter the feature attribution ranking beyond a tight Lipschitz bound.

Human-acceptable explanations that sound convincing but may not accurately represent the model's actual decision-making mechanics. A plausible rationale passes human review but fails faithfulness tests.

• Key Distinction: Plausibility is about human acceptance; faithfulness is about mechanistic accuracy
• Risk: Plausible but unfaithful rationales create a false sense of security and mask model errors
• Example: A loan denial explanation citing 'debt-to-income ratio' when the model actually relied on a spurious zip code correlation

The technique of generating an explanation after the model has made its decision, often using a secondary explainer model. This is the dominant paradigm in black-box explanation.

• Faithfulness Challenge: The explainer model may introduce its own biases, decoupling the rationale from the original model's logic
• Common Approaches: LIME, SHAP, and surrogate models all fall under post-hoc rationalization
• Verification Gap: Without access to the original model's internals, faithfulness can only be approximated

The degree to which a generated rationale accurately mirrors the true computational logic used by the model to arrive at a prediction. This is the core metric for faithful rationales.

• Measurement: Faithfulness metrics compare rationale-attributed features against the model's actual sensitivity to input perturbations
• Sufficiency vs. Comprehensiveness: A faithful rationale must identify both necessary and sufficient features for the prediction
• Trade-off: Highly faithful explanations are often less intuitive to humans than plausible ones

A technique that elicits step-by-step reasoning from large language models by providing few-shot examples of intermediate logical steps. The faithfulness of these chains is actively debated.

• Faithfulness Concern: Models may generate coherent reasoning that rationalizes a conclusion they reached through different, opaque means
• Self-Consistency: Sampling multiple reasoning paths and taking a majority vote can improve reliability
• Research Frontier: Mechanistic interpretability aims to verify whether CoT traces reflect actual model computation

Quantitative measures designed to automatically score how accurately a generated explanation reflects the model's internal decision process.

• Erasure-Based: Measures prediction change when 'explained' features are removed; a faithful rationale should identify features whose removal alters the output
• Comprehensiveness: Quantifies how much of the model's decision is captured by the explanation
• Sufficiency: Measures whether the explained features alone can reproduce the original prediction
• Limitation: All metrics are proxies; ground-truth faithfulness requires access to model internals

Models architected with an intrinsic explainability component that generates interpretations as part of the forward pass, rather than relying on post-hoc analysis.

• Faithfulness Advantage: Explanations are produced by the same computation that generates predictions, reducing the decoupling risk
• Architecture: Typically combines a concept encoder, a parametrizer, and an aggregation layer that produces both predictions and explanations
• Trade-off: May constrain model capacity compared to black-box architectures optimized solely for accuracy