Claim-by-Claim Source Verification Prompt Template

This prompt is designed for QA engineers and eval pipeline builders who need automated, claim-level fact-checking of RAG-generated answers before they reach users. It decomposes a generated answer into individual factual claims, then verifies each claim against the provided source passages. The output is a structured verdict per claim (supported, partially supported, contradicted, or no evidence) with exact evidence spans and a rationale. Use this prompt when you need granular, auditable verification rather than a single pass/fail score. It belongs in post-generation guardrail pipelines, regression test suites, and any workflow where unsupported claims must be caught and traced back to specific answer fragments.

This prompt requires two non-negotiable inputs: the full text of the generated answer and the complete set of retrieved source passages that were used to produce it. Without both, the verification is meaningless. The prompt is most effective when the source passages are chunked with stable identifiers (document IDs, chunk indices) so the output can reference exact evidence locations. It is not a replacement for retrieval quality monitoring—if the retriever failed to surface relevant documents, this prompt will correctly flag claims as unsupported, but the root cause is a retrieval gap, not a generation hallucination. For high-stakes domains (healthcare, legal, finance), always route outputs flagged as 'contradicted' or 'no evidence' to human review before any downstream action.

Do not use this prompt as a real-time guardrail in latency-sensitive user-facing flows unless you have budgeted for the additional model call. Claim decomposition plus per-claim verification is token-intensive and adds measurable latency. For real-time use, consider a lighter-weight pre-filter like the Zero-Evidence Claim Flagging Prompt first, then escalate ambiguous cases to this full verification prompt. Also avoid using this prompt when the answer contains primarily subjective or opinion-based content—the verification framework assumes factual claims that can be matched against evidence. For subjective content, use a different evaluation rubric. After running this prompt, always log the structured output alongside the answer and source passages for audit trails and regression testing.

This template is the core instruction set for your claim-by-claim verification pipeline. It instructs the model to first extract discrete, testable claims from a generated answer, then evaluate each claim against the provided source passages. The output is a structured JSON array where every claim receives a verdict, a supporting evidence span, and a confidence indicator. Copy the template below, replace the square-bracket placeholders with your actual inputs, and wire it into your evaluation harness.

textCopy
You are a factual verification engine. Your task is to decompose a generated answer into individual claims, then verify each claim against the provided source passages.

## INPUT
- Generated Answer: [ANSWER_TEXT]
- Source Passages: [SOURCE_PASSAGES]
- Verification Threshold: [THRESHOLD]
- Output Schema: [OUTPUT_SCHEMA]

## INSTRUCTIONS
1. **Claim Extraction**: Read the generated answer and decompose it into a list of discrete, testable factual claims. Each claim must be a self-contained statement that can be verified independently. Ignore opinions, stylistic language, and transitional phrases. Output exactly one claim per item.
2. **Source Verification**: For each claim, search the provided source passages for evidence that supports, contradicts, or is silent on the claim. A claim is "SUPPORTED" only if the source passage explicitly states the same fact. A claim is "CONTRADICTED" if a source passage explicitly states the opposite. A claim is "NO_EVIDENCE" if no source passage addresses it.
3. **Evidence Span**: For SUPPORTED and CONTRADICTED verdicts, quote the exact source text that provides the evidence. Include the source passage identifier if available.
4. **Confidence**: Assign a confidence score from 0.0 to 1.0 reflecting how certain the verdict is. Use 1.0 only when the evidence is unambiguous and directly on point.
5. **Abstention**: If the generated answer contains no testable factual claims, return an empty claims array with a note explaining why.

## OUTPUT FORMAT
Return a JSON object with the following structure:
{
  "claims": [
    {
      "claim_id": "string",
      "claim_text": "string",
      "verdict": "SUPPORTED | CONTRADICTED | NO_EVIDENCE",
      "evidence_span": "string | null",
      "source_id": "string | null",
      "confidence": number,
      "rationale": "string"
    }
  ],
  "summary": {
    "total_claims": number,
    "supported_count": number,
    "contradicted_count": number,
    "no_evidence_count": number,
    "overall_support_ratio": number
  }
}

## CONSTRAINTS
- Do not fabricate evidence. If no source supports a claim, the verdict must be NO_EVIDENCE.
- Do not merge claims. Each claim must be a single, atomic fact.
- Do not skip claims. Every factual statement in the answer must appear in the claims array.
- If the source passages are empty or insufficient, set all verdicts to NO_EVIDENCE and note the condition in the summary.

To adapt this template, replace [ANSWER_TEXT] with the generated answer your system produced, and [SOURCE_PASSAGES] with the retrieved context chunks, each labeled with a source identifier. Set [THRESHOLD] to the minimum confidence score your pipeline will accept before flagging a claim for human review. Replace [OUTPUT_SCHEMA] with your application's expected JSON structure if it differs from the default. For high-stakes domains like clinical or legal verification, add a [RISK_LEVEL] placeholder that triggers stricter confidence thresholds and mandatory human review for any CONTRADICTED or NO_EVIDENCE verdicts. Always run the output through a JSON schema validator before ingesting it into downstream systems, and log every verification result alongside the prompt version and model identifier for auditability.

The Claim-by-Claim Source Verification Prompt Template is designed to operate as a post-generation gate within a RAG pipeline, not as a standalone chat interaction. After the primary model generates an answer, the application should extract that answer along with the full set of retrieved source passages and pass them into this verification prompt. The prompt decomposes the answer into discrete claims and verifies each against the provided context. The structured output—a list of claims with verdicts and supporting evidence spans—becomes the input to downstream decision logic: pass the answer to the user, block it, flag it for review, or trigger a regeneration with stricter grounding instructions.

To integrate this prompt into a production system, wrap it in a validation layer that enforces the expected output schema before the verdict is trusted. The prompt should return a JSON object with a claims array where each element contains claim_text, verdict (one of SUPPORTED, PARTIALLY_SUPPORTED, UNSUPPORTED, CONTRADICTED), source_evidence_spans (array of exact text excerpts from the provided context), and rationale. Implement a schema validator that rejects malformed responses and triggers a retry with the same inputs and a stronger format reminder. If the validator fails three times, escalate the answer to a human review queue rather than silently passing it. Log every verification result—including the raw model output, validation errors, and retry count—so that prompt drift and model behavior changes become observable over time.

For high-stakes domains such as healthcare, legal, or financial compliance, add a severity classifier after claim verification. Claims marked UNSUPPORTED or CONTRADICTED should be scored by a secondary prompt that classifies the hallucination severity as CRITICAL, MODERATE, or MINOR. Configure your release gate so that any answer containing a CRITICAL hallucination is automatically blocked and routed for human review, while MINOR unsupported claims in non-regulated contexts may be surfaced to users with an uncertainty disclaimer. This tiered approach prevents the verification prompt from becoming a binary pass/fail switch that either blocks too aggressively or lets dangerous content through. Always run the verification prompt on the final answer text that the user would see—not on an intermediate draft—to catch hallucinations introduced during formatting or post-processing steps.