Reference-Guided Hallucination Detection Prompt Template

This prompt is designed for content verification teams, RAG system builders, and evaluation engineers who need a structured, claim-by-claim audit of AI-generated text against a known-correct reference document. The primary job-to-be-done is production hallucination auditing: you have an authoritative source and a model-generated output, and you need a machine-readable trail that marks every claim as grounded, hallucinated, or unsupported before the content is published, stored, or acted upon. The ideal user is someone integrating this into an automated pipeline—think a CI/CD step for documentation, a pre-publish check for AI-written summaries, or a regression gate for RAG answer quality.

This prompt is not a general factuality scorer, a toxicity filter, or a replacement for human editorial review in high-risk domains. It requires an explicit, comprehensive reference text. If the reference is incomplete, the prompt will correctly flag claims as 'unsupported,' but that label may reflect reference gaps rather than model errors. Use this when you can provide a reference that is authoritative enough to confirm or refute each claim. The prompt works best with factual, declarative content—summaries, reports, technical documentation, and RAG answers—rather than creative writing, opinion pieces, or open-ended analysis where 'ground truth' is ambiguous. For regulated domains such as healthcare, finance, or legal, this prompt is a triage tool, not a final arbiter; always route severity-flagged outputs to human review.

Before deploying this prompt, ensure you have three things: a model-generated text to audit, a reference document that contains the authoritative facts, and a clear policy for what happens to each severity classification. The prompt produces a structured JSON output with claim-level verdicts and severity labels, which you can feed directly into downstream validation logic, logging systems, or review queues. Avoid using this prompt on outputs longer than a few thousand words without chunking—the claim decomposition step loses reliability on very long texts. If you need aggregate metrics across many outputs, pair this prompt with the Regression Test Suite Execution or Golden Dataset Benchmark Harness playbooks. For real-time streaming outputs, consider a lighter-weight groundedness check instead.

This prompt template is the core of a reference-guided hallucination detection harness. It instructs the model to act as an auditor, decomposing the [MODEL_OUTPUT] into discrete claims and then checking each one against the [REFERENCE_TEXT]. The output is a structured JSON report that flags every claim as grounded, hallucinated, or unsupported, with a severity classification for any violations. This is not a general-purpose fact-checker; it strictly measures fidelity to the provided reference, not objective truth.

textCopy
You are an expert fact-checking auditor. Your task is to compare a piece of generated text against a provided reference document. You must identify every factual claim in the generated text and determine whether it is supported by the reference.

# INPUTS
## MODEL OUTPUT
[MODEL_OUTPUT]

## REFERENCE TEXT
[REFERENCE_TEXT]

# INSTRUCTIONS
1. Decompose the entire MODEL OUTPUT into a list of discrete, verifiable factual claims. A claim is a single statement that can be proven true or false against the REFERENCE TEXT. Ignore opinions, stylistic phrasing, and grammatical structures.
2. For each claim, search the REFERENCE TEXT for direct support.
3. Classify each claim into exactly one of the following categories:
   - **grounded**: The claim is explicitly stated in or directly inferable from the REFERENCE TEXT.
   - **hallucinated**: The claim directly contradicts information in the REFERENCE TEXT.
   - **unsupported**: The claim is not found in the REFERENCE TEXT and cannot be confirmed or denied by it.
4. For any `hallucinated` or `unsupported` claim, assign a severity:
   - **critical**: The claim introduces a dangerous, legally risky, or fundamentally misleading fabrication.
   - **major**: The claim is a significant factual error or unsupported assertion that changes the meaning.
   - **minor**: The claim is a small detail, date, or name that is wrong or unsupported but does not change the core meaning.
5. For every claim, provide the exact text span from the MODEL OUTPUT and, for `grounded` claims, the exact supporting text span from the REFERENCE TEXT.

# CONSTRAINTS
- Do not use outside knowledge. The REFERENCE TEXT is the only source of truth.
- If the MODEL OUTPUT contains no verifiable claims, return an empty list.
- If a claim is a summary or paraphrase, it is `grounded` only if the core fact is present in the REFERENCE TEXT.

# OUTPUT SCHEMA
Return a valid JSON object with the following structure:
{
  "analysis": [
    {
      "claim_text": "String - the exact claim from the model output",
      "category": "grounded | hallucinated | unsupported",
      "severity": "critical | major | minor | null",
      "reference_evidence": "String or null - the supporting text from the reference for grounded claims",
      "explanation": "String - brief reasoning for the classification"
    }
  ],
  "summary": {
    "total_claims": "Integer",
    "grounded_count": "Integer",
    "hallucinated_count": "Integer",
    "unsupported_count": "Integer"
  }
}

To adapt this template, replace [MODEL_OUTPUT] with the text you are auditing and [REFERENCE_TEXT] with the authoritative source. For high-stakes domains like healthcare or finance, always route critical and major findings to a human review queue. Before deploying, run this prompt against a golden dataset of known hallucinations to calibrate your severity thresholds and ensure the JSON output is reliably parseable by your downstream validation logic.

This prompt is designed to be the core analysis step in a content verification pipeline, not a standalone chat interaction. The typical harness wraps the LLM call in a pre-processing layer that extracts and normalizes claims from the generated content, then feeds each claim alongside the reference text into this prompt. Post-processing validates the structured output, enriches it with metadata, and routes high-severity hallucinations to a human review queue. The harness should treat the LLM as a claim classifier that produces machine-readable verdicts, not as a final arbiter of truth.

Input preparation requires splitting the generated content into discrete claims before invoking this prompt. Use a lightweight extraction step—either a separate LLM call with a claim extraction prompt or a rule-based sentence splitter—to produce a list of atomic statements. Each claim is then paired with the full reference text and passed to this prompt individually or in small batches. Output validation must parse the JSON response and verify that every claim in the input appears in the output with exactly one verdict from the allowed enum: grounded, hallucinated, or unsupported. Reject malformed responses and retry with the same input up to two times before escalating. Severity routing uses the severity field to decide next actions: low severity hallucinations can be logged for trend analysis, medium triggers a review task in a queue, and high or critical should block publication and notify the content owner immediately. Store the full prompt, response, and validation result in an audit log for traceability.

Model choice matters for this workflow. Use a model with strong instruction-following and JSON output capabilities—GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro are suitable defaults. Avoid smaller or older models that may conflate unsupported with hallucinated or fail to produce valid JSON under the output schema. Retry logic should distinguish between format failures (invalid JSON, missing fields) and content failures (verdicts that don't match the claim list). Format failures get up to two retries with the same prompt; content failures should be logged and escalated without retry, as they indicate a model capability gap rather than a transient error. Human review integration is essential for medium and above severity findings. The harness should enrich each flagged claim with the original content snippet, the reference excerpt, the model's reasoning, and a direct link to the source document so reviewers can make a final determination without reconstructing context. Track reviewer decisions to measure judge accuracy over time and feed corrections back into eval datasets for prompt improvement.