Rationale Consistency

The fundamental property that a model must not assign contradictory justifications to inputs that differ only in irrelevant attributes.

• Definition: If input A and input B are semantically identical for a given task, the generated rationales must not conflict.
• Example: A loan application model should not cite 'high income' as a positive factor for one applicant and 'high income' as a negative factor for another with an identical financial profile.
• Detection: Automated theorem provers and natural language inference (NLI) models are used to check for logical entailment and contradiction between rationale pairs.

The generated explanation should remain stable under small, semantically meaningless perturbations of the input.

• Synonym Substitution: Replacing words with synonyms (e.g., 'happy' to 'joyful') should not alter the core reasoning.
• Paraphrase Consistency: A rationale for a paraphrased query must preserve the same causal logic and cited evidence.
• Adversarial Robustness: Consistency metrics expose brittle models that change their stated reasoning when faced with minor typographical errors or irrelevant token insertions.

The internal logical flow of a single rationale must be well-formed and free of self-contradiction.

• Premise-Conclusion Alignment: The stated evidence must logically support the final conclusion. A rationale stating 'The patient has a fever, therefore the patient has a broken bone' lacks structural coherence.
• Temporal Consistency: In sequential reasoning, the order of events in the explanation must not violate causality.
• Metric: Graph-based dependency parsing can map the logical structure of a rationale to verify that all claims are connected and non-circular.

Explanations for similar predictions across a dataset should rely on consistent, generalizable principles rather than spurious correlations.

• Feature Overlap: For two similar positive sentiment predictions, the model should consistently point to positive phrases, not randomly alternate between 'great acting' and 'loud sound'.
• Distributional Consistency: The set of features cited as important should follow a stable distribution across a test set, not fluctuate wildly.
• Use Case: Auditing for algorithmic fairness requires cross-sample stability to ensure protected attributes are not implicitly used in contradictory ways.

Consistency is a necessary but insufficient condition for faithfulness. A model can be consistently wrong.

• Plausible but Unfaithful: A model might consistently generate a plausible-sounding rationale that has no causal link to its actual internal computation.
• Joint Measurement: High consistency combined with high faithfulness metrics (e.g., input erasure tests) provides strong evidence of genuine interpretability.
• Simulatability Test: If a human can use the consistent rationales to predict the model's output on new data, the explanations are both consistent and functionally useful.

Quantitative methods for measuring rationale consistency in production systems.

• Pairwise NLI Score: Use a natural language inference model to classify rationale pairs as 'entailment', 'neutral', or 'contradiction'. A high contradiction rate signals poor consistency.
• Semantic Textual Similarity (STS): Compute cosine similarity between rationale embeddings for paraphrased inputs. Low similarity indicates instability.
• Rationale Clustering: Cluster generated rationales for a class of inputs. A high number of disjoint clusters suggests the model lacks a unified reasoning policy.

The degree to which a generated rationale accurately mirrors the true computational logic used by the model. A faithful rationale reveals actual feature weights and decision boundaries, not a plausible post-hoc story.

• Directly impacts consistency: unfaithful rationales drift across similar inputs
• Measured via erasure tests and comprehensiveness metrics
• Contrasts with plausible rationales that sound correct but misrepresent model internals

A metric ensuring that rationale content does not contradict real-world knowledge or provided source data. Factually inconsistent explanations erode user trust even when logically coherent.

• Evaluated using natural language inference (NLI) models
• Critical for source-grounded explanation systems
• Detects extrinsic hallucinations where generated claims conflict with established facts

The ability of a human observer to use a model's explanation to correctly anticipate the model's output on a new, unseen input. Consistent rationales improve simulatability by establishing predictable patterns.

• Measured via forward simulation prediction tasks
• High simulatability correlates with mechanistic interpretability
• Tests whether explanations actually transfer understanding to users

A technique that elicits step-by-step reasoning from large language models by providing few-shot examples of intermediate logical steps. CoT outputs must maintain internal consistency across reasoning chains.

• Self-consistency decoding samples multiple reasoning paths and selects the majority conclusion
• Inconsistent chains indicate unfaithful reasoning or model uncertainty
• Foundation for verbalized uncertainty in rationale generation

Natural language descriptions of the minimal changes to an input that would have resulted in a different prediction. Consistency requires that counterfactuals remain stable under small perturbations.

• Tests logical coherence by verifying that stated changes actually flip predictions
• Related to contrastive explanations (why A instead of B)
• Essential for actionable explanations in recourse systems

Quantitative measures designed to automatically score how accurately a generated explanation reflects the model's internal decision process. These metrics directly evaluate rationale consistency across input neighborhoods.

• Sufficiency and comprehensiveness scores measure feature attribution quality
• Monotonicity checks verify that increasing important features consistently changes outputs
• Used to benchmark LLM-as-Explainer paradigms against ground-truth model behavior