Hallucination Score

Measures the cosine similarity between the embedding of the generated response and the embedding of the source ground-truth text. This technique, often using models like Sentence-BERT or proprietary embedding APIs, quantifies conceptual overlap rather than exact lexical matching.

• High Score: Indicates strong semantic alignment with the reference.
• Low Score: Suggests the agent has drifted into unrelated or fabricated territory.
• Limitation: Fails to detect subtle contradictions where the generated text is topically similar but factually inverted.

Utilizes a dedicated NLI classifier to determine if a source document logically entails the generated hypothesis. A hallucination is flagged when the output contradicts the premise or is classified as neutral.

• Entailment Score: Probability that the generation is strictly supported by the evidence.
• Contradiction Score: Probability that the generation directly opposes the source.
• Advantage: Detects factual inversions that semantic similarity misses, such as 'The patient has a fever' vs. 'The patient has no fever.'

Leverages the stochastic nature of LLMs by sampling multiple reasoning paths (temperature > 0) for the same prompt. The hallucination score is derived from the entropy or divergence of these outputs.

• Low Variance: Indicates high confidence and likely factual accuracy.
• High Variance: Suggests the model is 'guessing' between multiple hallucinated realities.
• Mechanism: Often uses semantic entropy to cluster equivalent answers before calculating variance, preventing penalization for valid paraphrasing.

In RAG architectures, the score evaluates if the generated answer is attributable to the specific retrieved chunks. It measures hallucination as the proportion of generated entities or claims that cannot be mapped back to the provided context.

• Context Precision: Verifies if the model used the right chunk.
• Context Recall: Verifies if the model missed crucial information.
• Faithfulness Metric: Calculated as (Number of claims supported by context) / (Total number of claims in response).

Applies statistical methods to produce a prediction set with a mathematical guarantee of correctness. Instead of a single output, the agent produces a set of plausible answers.

• Non-Hallucination Guarantee: The score represents the probability that the true answer lies within the generated set.
• Abstention Trigger: If the set size exceeds a threshold (e.g., the model must include 50 options to reach 95% confidence), the score is low, and the system abstains from answering.
• Key Differentiator: Provides a formal, distribution-free risk bound rather than a heuristic score.

An agentic workflow where the LLM generates an initial response, then independently drafts and answers a series of fact-checking questions based on its own output. The hallucination score is the ratio of verification questions that the model fails to answer consistently.

• Process: 1. Draft response. 2. Plan verifications. 3. Execute verifications. 4. Generate final revised output.
• Score Derivation: (Failed Verifications) / (Total Verification Questions).
• Utility: Highly effective for detecting closed-book hallucinations where no external retrieval is available.

A complementary uncertainty metric that measures meaning-level variance rather than token-level probability. Unlike naive perplexity, semantic entropy clusters generations by bidirectional entailment—if two outputs convey the same fact, they are grouped—then calculates entropy across these meaning clusters. This provides a more robust hallucination signal because an LLM can be highly confident in its token choices while being deeply uncertain about the underlying facts. Used alongside hallucination scores to catch confident confabulations that surface-level metrics miss.

A self-correcting prompting technique that directly reduces hallucination scores. The process unfolds in four stages:

• Draft: Generate an initial response to the query
• Plan: Formulate a list of independent verification questions targeting each factual claim
• Execute: Answer each verification question separately, without referencing the original draft
• Finalize: Revise the initial response to align with verified answers This method systematically exposes unsupported assertions and improves factual consistency without external tools.

The process of anchoring every factual claim in an agent's output to an explicit, retrievable source document. Citation grounding transforms hallucination scoring from a probabilistic estimate into a binary audit: either a claim has a supporting citation or it does not. Modern implementations use Natural Language Inference (NLI) models to verify that the cited passage actually entails the generated statement, preventing the common failure mode where agents cite irrelevant sources to appear credible. This is the gold standard for enterprise deployments requiring auditability.

A secondary LLM or classifier that evaluates the primary agent's outputs for factual consistency, forming the computational backbone of many hallucination scoring pipelines. Common architectures include:

• NLI-based critics: Fine-tuned on datasets like ANLI or FEVER to classify entailment vs. contradiction
• LLM-as-judge: A stronger model (e.g., GPT-4 evaluating GPT-3.5 outputs) that provides nuanced consistency ratings
• Cross-encoder rerankers: Models that score the alignment between a claim and its purported evidence source The critic's confidence score often serves directly as the hallucination score.

A statistical framework that moves beyond point estimates to measure how much a model 'knows that it doesn't know.' Key techniques relevant to hallucination scoring include:

• Conformal prediction: Produces prediction sets with guaranteed coverage probabilities, flagging outputs where the model cannot commit to a single answer
• Monte Carlo dropout: Runs multiple stochastic forward passes to estimate output variance
• Deep ensembles: Trains multiple models and measures disagreement as a proxy for uncertainty High uncertainty correlates strongly with hallucination risk, making these methods essential for selective prediction—knowing when to abstain.

A generation-time technique that prevents hallucinations by restricting the LLM's output to a predefined schema or grammar. Rather than detecting hallucinations post-hoc, constrained decoding makes certain categories of fabrication structurally impossible. Implementation methods include:

• Logit masking: Applying a boolean mask to invalid tokens before softmax, forcing the model to sample only from valid continuations
• Grammar-constrained sampling: Using finite-state automata to enforce JSON schemas, regular expressions, or domain-specific languages
• Retrieval-anchored generation: Forcing the model to only generate tokens found in retrieved context windows This approach eliminates syntactic hallucination but does not address semantic factual errors within valid structures.