Hallucination Detection Gate Prompt

This prompt acts as a binary gate for RAG pipelines, content verification systems, and any workflow where AI-generated text must be fact-checked against provided source documents. It decomposes a model's output into individual claims, maps each claim to specific evidence in the source material, and returns a grounded/ungrounded verdict. The primary job-to-be-done is preventing hallucinated content from reaching end users by enforcing evidence-backed output before delivery. Ideal users include RAG system builders, evaluation engineers running CI/CD prompt regression suites, and production operators who need continuous hallucination monitoring on live traffic.

Deploy this prompt as a pre-delivery check in question-answering systems where the cost of a hallucination is high—customer-facing support, clinical documentation, legal research, or financial analysis. Wire it into your evaluation harness as a regression gate: every prompt change must pass hallucination detection against a golden dataset of known grounded and ungrounded output pairs. For production monitoring, sample a percentage of live responses through this gate and track the grounded-to-ungrounded ratio over time. A rising ungrounded rate signals retrieval degradation, prompt drift, or model behavior change that requires investigation before users notice.

Do not use this prompt when no source evidence is provided, when the output is purely creative or subjective, or when the acceptable risk of hallucination is non-zero and the cost of blocking is higher than the cost of an error. This prompt is also inappropriate for open-domain conversation where grounding against a fixed document set would produce false negatives on legitimate general-knowledge responses. If your workflow requires nuanced faithfulness scoring rather than a binary gate, use the RAG Answer Faithfulness Gate Prompt instead. For production deployments, always pair this prompt with a human review escalation path for edge cases where the model's grounding verdict is low-confidence or where blocked outputs represent high-value transactions that warrant manual verification.

This prompt template implements a hallucination detection gate for Retrieval-Augmented Generation (RAG) systems. It takes a model-generated response and the source documents used to produce it, then evaluates every factual claim individually. The prompt is designed to be placed in your system instructions for an LLM judge that acts as a verification layer before outputs reach end users. Replace the square-bracket placeholders with your actual data before sending the request.

textCopy
You are a hallucination detection judge for a RAG system. Your job is to verify whether every factual claim in a generated response is supported by the provided source documents.

## INPUT
- Generated Response: [RESPONSE]
- Source Documents: [SOURCE_DOCUMENTS]

## INSTRUCTIONS
1. Extract every factual claim from the Generated Response. A factual claim is any statement that asserts something true about the world, including dates, names, numbers, events, relationships, or properties.
2. For each claim, search the Source Documents for direct supporting evidence. A claim is grounded only if the source documents explicitly state it or it can be directly inferred from a single source without adding external knowledge.
3. Do not mark a claim as grounded if:
   - The source only partially supports it.
   - The claim combines information from multiple sources in a way not present in any single source.
   - The claim adds details, interpretations, or context not present in the sources.
   - The claim contradicts the sources.
4. For each claim, provide a direct quote from the source documents that supports it, or state "NO EVIDENCE FOUND" if no support exists.

## OUTPUT FORMAT
Return a JSON object with this exact structure:
{
  "overall_verdict": "grounded" | "ungrounded",
  "claims": [
    {
      "claim_text": "string",
      "verdict": "grounded" | "ungrounded",
      "source_quote": "string or null",
      "source_id": "string or null"
    }
  ]
}

## CONSTRAINTS
- overall_verdict must be "ungrounded" if ANY claim is ungrounded.
- Never fabricate source quotes. If no evidence exists, use null for source_quote and source_id.
- If the Generated Response contains no factual claims, return an empty claims array with overall_verdict "grounded".
- Do not evaluate claims about the response's own structure, formatting, or style.

To adapt this prompt, replace [RESPONSE] with the full text output from your generation model and [SOURCE_DOCUMENTS] with the retrieved passages, each labeled with a source identifier. If your sources lack IDs, add them before calling the judge. For high-stakes domains such as healthcare or legal, add a [RISK_LEVEL] parameter that triggers stricter evidence requirements and mandatory human review when the verdict is ungrounded. Test the prompt with both clearly grounded responses and known hallucinations to calibrate your acceptance threshold before deploying to production.

The hallucination detection gate operates as a synchronous check between answer generation and user delivery. After your RAG pipeline produces a candidate answer, split it into individual claims using sentence segmentation or a lightweight extraction step. Send each claim alongside its source context to the gate prompt. The gate returns a binary grounded/ungrounded verdict per claim with an evidence mapping. Aggregate these verdicts to decide whether the full answer passes, fails, or requires partial redaction. This gate is not a replacement for retrieval quality—it catches fabrication after generation, so pair it with retrieval precision monitoring.

Wire the gate as a post-generation validation step in your orchestration layer. For each claim, call the prompt with [CLAIM], [SOURCE_CONTEXT], and [EVIDENCE_THRESHOLD] set to your acceptable grounding level. Parse the JSON output and check the verdict field. If any claim returns ungrounded, either redact that claim, regenerate the answer with stricter grounding instructions, or escalate to human review depending on your risk tolerance. Implement a retry loop with a maximum of two regeneration attempts—after that, log the failure and surface the ungrounded claims to an operator. For high-stakes domains like healthcare or legal, configure the harness to block delivery entirely when any claim fails grounding and require human approval before release.

Log every gate invocation with the claim text, source context, verdict, and evidence mapping. This audit trail is essential for debugging retrieval gaps, tuning the evidence threshold, and defending decisions to stakeholders. Set up eval checks that run weekly against a golden dataset of known grounded and hallucinated claims. Measure false negatives on paraphrased support—where the model marks a claim ungrounded despite the source containing equivalent meaning—and false positives on plausible-sounding fabrications. If false negatives exceed 5%, lower the evidence threshold or add few-shot examples of acceptable paraphrasing. If false positives exceed 2%, tighten the threshold and add examples of subtle fabrications. Use these metrics to decide when to update the prompt template or switch to a more capable model for the gate itself.