RAG Hallucination Detection via Evidence Check Prompt

This prompt is designed for verification teams and RAG pipeline operators who need a systematic, repeatable audit of generated answers against their source documents, not a one-off spot check. The core job-to-be-done is to prevent hallucinated or unsupported information from reaching end-users by inserting a structured verification layer after answer synthesis but before the response is returned. The ideal user is an engineer or operator responsible for output quality in a production system where factual accuracy is non-negotiable, such as in legal research, medical literature review, or financial compliance. The required context is a fully generated answer and the exact set of source passages used during retrieval-augmented generation; without both, the prompt cannot perform its function.

Use this prompt when you need to decompose a generated answer into discrete factual statements and classify each as supported, contradicted, or unsupported by the provided evidence. It is appropriate for workflows where a false positive (flagging a correct paraphrase as a hallucination) is less damaging than a false negative (missing a fabricated claim), but you must still calibrate its strictness. This prompt belongs in a post-generation verification layer, wired as a synchronous check or an asynchronous audit step. It is particularly valuable when you are iterating on retrieval quality or model behavior and need a consistent measurement of factual grounding over time, rather than relying on spot-checking or aggregate user feedback.

Do not use this prompt as a substitute for retrieval quality improvements or as a real-time guardrail in latency-sensitive chat applications where the added inference time would degrade user experience. It is also not designed for cases where the source documents themselves contain conflicting information—use a contradiction detection prompt for that upstream task. If your application requires legal or clinical certification of accuracy, this prompt's output should be treated as a preliminary audit that requires human review before any binding decision. The next step after reading this section is to copy the prompt template, adapt the output schema to your logging infrastructure, and run it against a golden dataset of known hallucinations and correct answers to establish your pass/fail thresholds before production deployment.

This prompt template is designed to be dropped directly into your verification pipeline. It takes a generated answer and the source documents used to produce it, then outputs a structured JSON report flagging unsupported claims, identifying the specific spans of text that lack evidence, and documenting what evidence is missing. The prompt includes explicit guardrails against the common failure mode of flagging paraphrased or reworded support as hallucination, which is critical for maintaining user trust in your RAG system's verification layer.

textCopy
You are an expert fact-checker evaluating a generated answer against provided source documents. Your task is to identify claims in the answer that are not supported by the evidence.

## INPUT

**Generated Answer:**
[GENERATED_ANSWER]

**Source Documents:**
[SOURCE_DOCUMENTS]

## INSTRUCTIONS

1. Decompose the generated answer into discrete, verifiable claims.
2. For each claim, search the source documents for supporting evidence.
3. A claim is SUPPORTED if the source documents contain the same factual information, even if the wording differs. Paraphrasing, synonym use, and rewording do NOT constitute hallucination.
4. A claim is UNSUPPORTED only if the source documents lack the factual content entirely, contradict it, or provide insufficient information to verify it.
5. For each unsupported claim, identify the exact text span in the generated answer and explain what evidence is missing.
6. If the source documents are insufficient to verify a claim but do not contradict it, mark it as INSUFFICIENT_EVIDENCE rather than UNSUPPORTED.

## OUTPUT SCHEMA

Return a JSON object with the following structure:
{
  "overall_assessment": {
    "total_claims": <integer>,
    "supported_claims": <integer>,
    "unsupported_claims": <integer>,
    "insufficient_evidence_claims": <integer>,
    "hallucination_ratio": <float between 0 and 1>
  },
  "claims": [
    {
      "claim_id": <integer>,
      "claim_text": "<exact text from answer>",
      "answer_span": "<character offsets or quote>",
      "verdict": "SUPPORTED | UNSUPPORTED | INSUFFICIENT_EVIDENCE",
      "supporting_source": "<quote from source documents or null>",
      "source_location": "<document identifier and section or null>",
      "missing_evidence_description": "<what evidence would be needed or null>",
      "confidence": "HIGH | MEDIUM | LOW"
    }
  ]
}

## CONSTRAINTS

- Do NOT flag a claim as unsupported solely because the source uses different words. Semantic equivalence counts as support.
- If multiple source documents address the same claim, cite the strongest supporting source.
- If sources contradict each other on a claim, note the contradiction and mark the claim as UNSUPPORTED with an explanation.
- Assign LOW confidence to any verdict where the evidence is ambiguous or incomplete.
- If the generated answer contains no verifiable factual claims, return an empty claims array with appropriate overall_assessment values.

To adapt this prompt for your pipeline, replace [GENERATED_ANSWER] with the full text output from your RAG system and [SOURCE_DOCUMENTS] with the retrieved passages, including document identifiers and section metadata. The output schema is designed for direct parsing into your verification database. If your RAG system uses chunk IDs or paragraph indices, add those to the source_location field for precise provenance tracking. For high-stakes domains like healthcare or legal, add a [RISK_LEVEL] parameter that adjusts the confidence thresholds and requires human review for any UNSUPPORTED verdicts.

Before deploying, test this prompt against a golden dataset that includes both genuine hallucinations and correctly paraphrased support. The most common failure mode is over-flagging—marking reworded but accurate statements as unsupported. If your eval shows high false-positive rates, reinforce the paraphrasing instruction by adding few-shot examples showing acceptable rewording versus actual fabrication. For production use, always log the full prompt response alongside the original answer and sources to enable retrospective audit and prompt iteration.