Hallucination Detection

This method uses specialized NLI models (e.g., DeBERTa, RoBERTa fine-tuned on MNLI) to perform a formal logical check between a generated claim and its source context. The model classifies the relationship as:

• Entailment: The source context supports the claim.
• Contradiction: The source context directly contradicts the claim.
• Neutral: The source context provides insufficient information to verify the claim. This provides a probabilistic score for factual consistency, often used as a primary verification layer in production RAG systems.

This technique measures the semantic overlap between the generated answer and the retrieved source passages, independent of exact word matching. It uses:

• Embedding Similarity: Cosine similarity between dense vector representations (e.g., from Sentence-BERT) of the answer and the top-k retrieved chunks. Low similarity scores indicate potential hallucination.
• Lexical Overlap Metrics: Traditional metrics like ROUGE or BLEU, though less robust for meaning, can flag answers with zero keyword overlap with sources.
• Entity Consistency Check: Extracts named entities (people, places, dates) from both the answer and context; missing or conflicting entities are strong hallucination signals.

For complex, multi-fact answers, this method breaks the final output into individual, atomic claims using a decomposition model or simple heuristics. Each atomic claim is then independently verified against the knowledge base. This is critical for detecting composite hallucinations, where an answer mixes correct and incorrect facts. The process involves:

1. Decompose: "The Eiffel Tower is in Paris and was built in 1801." → [Claim A: Location], [Claim B: Build Date].
2. Retrieve: Search for evidence for each claim.
3. Verify: Apply NLI or similarity scoring per claim. A single false atomic claim fails the entire answer.

This leverages the stochastic nature of LLMs to detect instability indicative of hallucination. Key approaches include:

• Self-Consistency: Sampling multiple reasoning paths or answers (via temperature >0). If the model produces highly divergent factual answers, it indicates low confidence and high hallucination risk. The most frequent answer is often the most reliable.
• Model Ensembling: Using multiple, differently trained verification models (e.g., different NLI models, different embedding models) to score an answer. High variance in verification scores across the ensemble signals uncertainty and potential hallucination.
• Monte Carlo Dropout: Applying dropout at inference time to generate a distribution of confidence scores for a claim; a wide distribution suggests the model is guessing.

This technique focuses on interpreting the model's own internal confidence signals. LLMs often produce high softmax probabilities for hallucinated content. Calibration aims to align these probabilities with actual empirical accuracy. Methods include:

• Platt Scaling / Temperature Scaling: Post-processing the model's logits to produce better-calibrated confidence scores.
• Selective Prediction: Setting a confidence threshold. Answers generated with confidence below the threshold are automatically flagged for review or trigger an abstention. Metrics like Expected Calibration Error (ECE) quantify how well confidence predicts accuracy.
• Verbalized Confidence: Prompting the model to output its uncertainty in natural language (e.g., "I'm not certain, but..."), which can be parsed as a heuristic signal.

Simple, deterministic rules provide a fast, reliable first pass for detecting specific hallucination patterns. These are often implemented as a pre-filter before more costly ML-based verification. Examples include:

• Numerical Consistency: Checking if dates, statistics, or counts in the answer are present and match the context.
• Citation-Grounding Check: Ensuring every markdown citation [1] in the answer corresponds to a valid retrieved document ID.
• Out-of-Context Phrase Detection: Flagging highly specific technical terms or proper nouns that do not appear in the retrieved context at all.
• Contradiction Triggers: Searching for phrases like "there is no evidence" or "it is not known" in the context that contradict affirmative claims in the answer.

A faithfulness metric is a quantitative score that measures the degree to which a language model's generated output is factually consistent with and logically entailed by its provided source context. It is a core evaluation for RAG systems.

• Common Implementations: Often calculated using Natural Language Inference (NLI) models to judge if the answer is entailed by the context, or by comparing answer-context overlap using semantic similarity.
• Purpose: Provides an objective, automated benchmark for hallucination detection, allowing engineers to compare different prompting strategies, retrieval methods, or model versions.

Confidence calibration is the process of adjusting a model's internal probability scores so they accurately reflect the true likelihood of an output being correct. A well-calibrated model that is 90% confident should be correct 90% of the time.

• Importance for Detection: Poor calibration makes a model's own confidence scores useless for hallucination detection. Calibration enables reliable uncertainty quantification.
• Calibration Error: Measured by metrics like Expected Calibration Error (ECE), which quantifies the gap between predicted confidence and empirical accuracy. High ECE indicates unreliable self-assessment.

NLI-based verification uses a Natural Language Inference model as a dedicated fact-checker. It treats a generated claim as a hypothesis and the retrieved source context as a premise, classifying the relationship as entailment, contradiction, or neutral.

• Mechanism: This is a common post-hoc verification technique. The NLI model (e.g., DeBERTa, RoBERTa fine-tuned on MNLI) operates separately from the primary generator.
• Output: Provides a scalar score or label for each claim, enabling automated contradiction detection and filtering of unentailed statements.

Answer grounding is the technique of explicitly constraining a language model's generation to be directly derived from the retrieved source context. It's a preventative design pattern, not just a detection method.

• Implementation: Achieved through grounding prompting (e.g., "Answer only using the provided context"), constrained decoding, or architectures that force the model to copy tokens from the context.
• Goal: Minimizes the model's reliance on parametric memory, reducing the risk of hallucination by design. It enforces verifiable generation.

Selective answering is a strategy where a model is trained or prompted to only respond to queries it can answer with high confidence. A refusal mechanism is the specific function that declines to answer.

• Trigger: Activated when confidence scores are below a confidence threshold, when retrieved context is insufficient, or when a fact-checking module finds low support.
• Output: The model provides an abstention signal (e.g., "I cannot answer based on the provided information"). This is a critical safety feature to avoid speculative hallucinations.

Provenance tracking is the systematic recording of the origin and lineage of data used in generation. An audit trail is the immutable record produced, essential for root-cause analysis in hallucination detection.

• Contents: Includes the exact retrieved document chunks (source attribution), their source reliability scores, the model's prompt, confidence scores, and results of any post-hoc verification checks.
• Enterprise Value: Enables engineers to debug hallucinations by tracing which source led to an error and is mandatory for compliance in regulated industries.