Factual Consistency

Uses a secondary Natural Language Inference (NLI) model to check if the source document logically entails the generated rationale. If the premise (source) contradicts the hypothesis (rationale), the output is flagged as factually inconsistent.

• Direction: Source → Rationale
• Key Metric: Entailment score > 0.9 threshold
• Example: Source says 'Revenue grew 12%' but rationale claims 'Revenue doubled' — flagged as contradiction

Measures the percentage of atomic claims within a generated rationale that can be directly mapped to a specific span of text in the source material. Ungrounded claims are treated as potential hallucinations.

• Atomic Claim: Smallest verifiable fact unit
• Precision Formula: (Grounded Claims / Total Claims) × 100
• Target: > 95% grounding precision for production systems

Validates generated statements against a structured enterprise knowledge graph to detect contradictions with established facts. The system queries the graph for each extracted triple (subject-predicate-object) and flags mismatches.

• Triple Extraction: NLP parses rationale into RDF triples
• Consistency Check: SPARQL query verifies each triple
• Use Case: Preventing contradiction of regulatory or product specifications

Evaluates whether multiple rationales generated for semantically similar inputs produce logically compatible explanations. Inconsistency across paraphrased inputs indicates brittle reasoning rather than robust understanding.

• Method: Generate rationales for 5-10 paraphrased inputs
• Metric: Pairwise contradiction rate between outputs
• Goal: < 2% contradiction rate across paraphrases

Specifically targets time-sensitive claims by comparing generated dates, sequences, and event ordering against a temporal knowledge base. Prevents anachronisms where a rationale references events that hadn't occurred at the stated time.

• Temporal Extraction: Identify all date-anchored claims
• Validation Source: Time-stamped knowledge base entries
• Critical For: Financial reports, legal documents, medical histories

Automated extraction and cross-validation of all quantitative values in the rationale against source data. Detects fabricated statistics, incorrect units, and arithmetic errors in derived calculations.

• Extraction: Regex + NER for numbers and units
• Validation: Compare against source document values
• Tolerance: ±0.1% for financial figures, exact match for counts

Techniques used to identify and flag generated explanations that contain fabricated, nonsensical, or unfaithful information. Detection methods include:

• SelfCheckGPT: Sampling multiple responses to measure information consistency
• NER-based verification: Checking if named entities in the rationale exist in the source
• Entailment scoring: Measuring whether source documents logically support each claim

Hallucination detection serves as the primary gatekeeper for factual consistency, catching contradictions before they reach end-users.

The process of linking claims within a generated rationale directly to verifiable external documents or specific training data points. Key approaches include:

• Span-level attribution: Mapping each factual assertion to exact text segments
• Document retrieval verification: Confirming cited documents actually contain the claimed information
• Knowledge graph anchoring: Validating claims against structured, curated knowledge bases

Source grounding transforms a plausible-sounding rationale into an auditable, evidence-backed justification.

Quantitative measures designed to automatically score how accurately a generated explanation reflects the model's internal decision process. Common metrics include:

• SUFFICIENCY: Measuring if removing non-attributed input features changes the prediction
• COMPREHENSIVENESS: Assessing if attributed features alone are enough to maintain the prediction
• Correlation with attention weights: Comparing generated rationales against internal model focus

These metrics provide the objective benchmarks needed to evaluate factual consistency at scale.

The mechanism of grounding generated explanations by explicitly pointing to specific segments of the source input data as proof. Implementation strategies include:

• Token-level highlighting: Marking which input tokens support each output statement
• Cross-attention analysis: Tracing how decoder outputs attend to encoder representations
• Retrieval index mapping: Linking generated claims to chunk IDs in a vector database

Evidence attribution creates an unbroken chain of custody from model output back to source data.

The automated creation of precise references to source documents that support the factual assertions made in a model's explanation. Modern approaches include:

• ALCE benchmark: Evaluating models on citation accuracy and recall
• Post-hoc retrieval verification: Checking if cited passages actually entail the generated claim
• In-line citation formatting: Embedding references directly within generated text for traceability

Accurate citation generation is the gold standard for demonstrating factual consistency in enterprise deployments.

A metric evaluating whether a model generates logically coherent and non-contradictory explanations across similar inputs. Testing frameworks examine:

• Input perturbation stability: Whether small input changes produce proportionally small rationale changes
• Semantic equivalence preservation: If paraphrased inputs yield semantically identical explanations
• Cross-sample contradiction detection: Flagging when Model explains similar cases with conflicting logic

Consistency metrics catch subtle factual drift that individual hallucination checks might miss.