Hallucination Detection Eval Prompt Template

This prompt is designed for RAG and summarization teams who need to verify the factual accuracy of AI-generated text against a set of provided source documents. It instructs an LLM to act as an evaluator, decomposing a generated answer into individual claims and checking each one for support in the evidence. The output is a structured, claim-by-claim verification report that flags unsupported statements, fabricated details, and contradictions with the source material. Use this prompt as part of your regression testing suite to catch hallucination regressions before they reach production, or embed it in your CI/CD pipeline as a quality gate for any system where factual grounding is non-negotiable.

This prompt is ideal when you have a closed-world assumption: the source documents provided in [CONTEXT] represent the complete universe of acceptable evidence. It works best for extractive or abstractive summarization, RAG-based Q&A, and document-grounded content generation. The evaluator will flag any claim that cannot be directly attributed to the provided sources, making it particularly effective for catching fabricated statistics, invented quotes, and plausible-sounding but unsupported assertions. For best results, pair this with a golden dataset of known hallucinations and correct answers so you can measure both precision (are flagged claims actually unsupported?) and recall (are all hallucinations caught?).

Do not use this prompt when the generated text is expected to contain novel reasoning, creative synthesis, or external knowledge beyond the provided sources—the evaluator will incorrectly flag legitimate inferences as hallucinations. It is also unsuitable for evaluating stylistic qualities, tone, or user experience; this prompt checks only factual grounding. For open-ended generation tasks, prefer a reference-free evaluation prompt. For tasks requiring both factual and qualitative assessment, combine this with a multi-axis rubric scorer. Always require human review for high-stakes domains like healthcare, legal, or finance, where a missed hallucination could cause real harm. The LLM judge itself can hallucinate, so run this eval multiple times and check for score stability before trusting a single pass.

This prompt template is designed to be copied directly into your evaluation harness. It instructs an LLM judge to decompose a generated response into individual claims, then verify each claim against provided source context. The output is a structured report that flags unsupported statements, fabricated details, and source contradictions. Use this as the core instruction set for your hallucination detection eval step before shipping RAG or summarization features.

textCopy
You are an expert fact-checker evaluating AI-generated text for hallucinations. Your task is to verify whether every factual claim in the generated response is supported by the provided source context.

## INPUT

**Generated Response:**
[RESPONSE]

**Source Context:**
[SOURCE_CONTEXT]

## INSTRUCTIONS

1. Decompose the Generated Response into a list of discrete, verifiable factual claims. Ignore opinions, stylistic phrasing, and grammatical structures.
2. For each claim, search the Source Context for direct supporting evidence, indirect supporting evidence, or contradictions.
3. Classify each claim into exactly one of these categories:
   - **SUPPORTED**: The claim is directly stated in or can be directly inferred from the Source Context.
   - **UNSUPPORTED**: The claim is not present in the Source Context and cannot be inferred from it. This is a potential hallucination.
   - **CONTRADICTED**: The claim directly conflicts with information in the Source Context.
4. For SUPPORTED claims, include the exact quote or passage from the Source Context that supports it.
5. For UNSUPPORTED claims, note whether the claim is plausible but unverifiable, or likely fabricated.
6. For CONTRADICTED claims, include the conflicting evidence from the Source Context.
7. Calculate a **Groundedness Score** as: (Number of SUPPORTED claims) / (Total number of claims). Express as a percentage.

## OUTPUT FORMAT

Return a valid JSON object with this exact schema:

{
  "groundedness_score": number,
  "total_claims": number,
  "supported_claims": number,
  "unsupported_claims": number,
  "contradicted_claims": number,
  "claims": [
    {
      "claim_text": "string",
      "classification": "SUPPORTED" | "UNSUPPORTED" | "CONTRADICTED",
      "source_evidence": "string or null",
      "notes": "string"
    }
  ],
  "overall_assessment": "string"
}

## CONSTRAINTS

- Do not classify stylistic choices, tone, or formatting as claims.
- If the Generated Response contains no factual claims, return an empty claims array with a groundedness_score of 100 and note this in the overall_assessment.
- If the Source Context is empty or irrelevant, classify all factual claims as UNSUPPORTED.
- Be strict: if evidence is missing, do not assume it exists elsewhere.

To adapt this template, replace [RESPONSE] with the AI-generated text under evaluation and [SOURCE_CONTEXT] with the retrieved passages, document excerpts, or knowledge base entries that should ground the response. For RAG systems, [SOURCE_CONTEXT] is typically the chunks returned by your retriever. For summarization, it is the original document. If your eval harness requires a different output schema, modify the JSON structure but preserve the claim-level granularity—this is what makes the report actionable. Always run this eval against a golden dataset with known hallucinations before relying on it in CI/CD.

The hallucination detection eval prompt is not a one-off manual check; it is a programmatic component of a regression testing harness. Wire it into your CI/CD pipeline or a scheduled batch job that runs against every RAG or summarization output before release. The harness must supply the prompt with a structured [INPUT] containing the original query, the generated text, and the retrieved context or source documents. The model choice matters: use a capable instruction-following model (e.g., GPT-4, Claude 3.5 Sonnet) as the judge, and pin the model version to avoid judge drift across eval runs. If your application is high-stakes (healthcare, legal, finance), route all outputs flagged as containing unsupported claims to a human review queue before the eval result is considered final.

Build a validation layer around the prompt's output. The expected output is a JSON object with a claims array, where each claim has claim_text, verdict (one of supported, unsupported, contradicted, unverifiable), evidence_source, and confidence. Before accepting the eval result, validate that the JSON parses correctly, that every claim from the generated text is accounted for, and that no verdict values fall outside the allowed enum. If validation fails, retry the eval prompt once with the validation error message appended to the [CONSTRAINTS] field. Log every eval result—including raw model output, parse errors, and retry counts—to your observability platform for trace analysis. For RAG systems, cross-reference the evidence_source fields against your actual retrieved document IDs to catch cases where the judge hallucinates source references.

Set explicit pass/fail thresholds based on your risk tolerance. For example, a single contradicted claim or more than two unsupported claims might trigger an automatic failure and block the release. Use the confidence scores to weight findings: low-confidence flags should still surface for review but might not block deployment if all high-confidence verdicts are clean. Run the eval prompt against a golden dataset of known hallucinations and known faithful outputs to calibrate your thresholds before relying on it in production. Avoid treating the judge's output as ground truth—it is a signal that reduces, but does not eliminate, the need for spot-checking and human oversight in regulated domains.