Plausible Rationales

Explanations that accurately represent the model's true internal reasoning process. Unlike plausible rationales, faithful rationales do not fabricate a convincing but false story. They require access to the model's actual computational graph, attention weights, or activation patterns. A faithful rationale for a loan denial might correctly identify a specific interaction between debt-to-income ratio and credit history length, even if a simpler, more intuitive explanation exists.

The technique of generating an explanation after the model has made its decision, often using a secondary explainer model. This is the primary mechanism for creating plausible rationales. The explainer model observes the input and output, then constructs a narrative that correlates with the prediction without necessarily accessing the original model's internal logic. This introduces a critical fidelity gap between what the model did and what the explanation says it did.

A quantitative metric measuring the degree to which a generated rationale mirrors the true computational logic of the model. A faithfulness score of 0.9 indicates the explanation aligns with the model's actual decision boundary 90% of the time. Plausible rationales often score low on faithfulness metrics because they optimize for human comprehensibility rather than mechanistic accuracy. Key measurement techniques include:

The ability of a human observer to use a model's explanation to correctly anticipate the model's output on a new, unseen input. A highly simulatable explanation allows a user to mentally run the model's logic. Plausible rationales can be dangerously simulatable—they create a false mental model that leads users to confidently predict outcomes, only to be wrong when the model's actual, unstated logic diverges from the narrative.

A technique that elicits step-by-step reasoning from large language models by providing few-shot examples of intermediate logical steps. When used for explanation, the model verbalizes its reasoning process. However, research shows these verbalized chains can be plausible but unfaithful—the model may generate a coherent logical narrative that influenced its final answer far less than the tokens suggest, or not at all.

Techniques used to identify and flag generated explanations that contain fabricated, nonsensical, or unfaithful information. In the context of plausible rationales, hallucination detection must distinguish between two failure modes: factual hallucinations (citing non-existent evidence) and fidelity hallucinations (describing a reasoning process the model did not actually perform). The latter is significantly harder to detect automatically.