Reference-Backed Claim Verification Prompt Template

This prompt is built for teams embedding automated verification into content pipelines where every factual assertion must be traced to a source. Use it when you have a model-generated output or human-authored content that needs claim-by-claim auditing against a known reference document. The ideal user is an AI engineer, QA lead, or compliance reviewer who is integrating a verification gate into a CI/CD pipeline, a content management system, or a regression test suite. The required context is a complete, authoritative reference document and a target text to audit. The job-to-be-done is not to generate new answers, but to produce a structured, evidence-backed audit report that measures deviation from a known source of truth.

This is not a RAG answer-generation prompt. It is a post-hoc verification tool. Do not use it when you need the model to synthesize an answer from multiple retrieved documents, when no single authoritative reference exists, or when the goal is open-ended creative generation. It is also a poor fit for real-time chat moderation where latency constraints preclude a detailed claim-by-claim analysis. If your workflow requires evaluating whether a response is helpful, tonally appropriate, or stylistically consistent without a ground-truth reference, use a rubric-based LLM judge instead. This prompt assumes that deviation from the reference is measurable and that unsupported claims are failures.

Before wiring this into a production pipeline, ensure you have a reliable method for providing the reference document in the prompt context. If the reference is too long for a single context window, you must implement a chunking and reconciliation strategy outside the prompt. The output of this prompt should feed into a structured logging system, a human review queue for flagged claims, or an automated gate that blocks publication when the confidence score falls below a defined threshold. Start by running this against a golden dataset of known-correct and known-incorrect claim sets to calibrate your confidence thresholds before relying on it in production.

This template is designed to be dropped directly into your evaluation harness. It instructs the model to extract every factual claim from a target text, locate supporting or contradicting evidence in a provided reference, and produce a structured audit report. The output includes per-claim verdicts, explicit source spans, confidence scores, and a clear flag for claims that require human review. Replace every square-bracket placeholder with your actual inputs before sending this to the model. The template assumes you have already retrieved or provided the reference material; it does not perform retrieval itself.

textCopy
You are a meticulous claim verification auditor. Your task is to compare a target text against a reference document and produce a structured audit report.

## INPUT

**Target Text:**
[INPUT_TEXT]

**Reference Document:**
[REFERENCE_TEXT]

## INSTRUCTIONS

1. Extract every discrete factual claim from the Target Text. A claim is a statement that can be verified as true, false, or unverifiable against the Reference Document.
2. For each claim, search the Reference Document for evidence that supports, contradicts, or fails to address the claim.
3. Assign a verdict to each claim:
   - **SUPPORTED**: The Reference Document contains explicit evidence confirming the claim.
   - **CONTRADICTED**: The Reference Document contains explicit evidence that conflicts with the claim.
   - **UNVERIFIABLE**: The Reference Document does not contain sufficient evidence to confirm or contradict the claim.
4. Assign a confidence score from 0.0 to 1.0 for each verdict, where 1.0 means the evidence is unambiguous and directly on point.
5. For SUPPORTED and CONTRADICTED claims, include the exact source span from the Reference Document that serves as evidence.
6. Flag any claim for human review if:
   - The confidence score is below [CONFIDENCE_THRESHOLD]
   - The claim involves [HIGH_RISK_DOMAINS]
   - The evidence is ambiguous or requires specialized interpretation

## OUTPUT FORMAT

Return a JSON object with the following structure:

{
  "audit": {
    "target_text_id": "[TEXT_ID]",
    "reference_document_id": "[REF_ID]",
    "total_claims_extracted": <integer>,
    "claims": [
      {
        "claim_id": <integer>,
        "claim_text": "<exact claim from target text>",
        "verdict": "SUPPORTED | CONTRADICTED | UNVERIFIABLE",
        "confidence": <float 0.0-1.0>,
        "evidence_span": "<exact quote from reference> | null",
        "evidence_location": "<section or paragraph reference> | null",
        "reasoning": "<brief explanation of verdict>",
        "requires_human_review": <boolean>,
        "human_review_reason": "<reason if flagged> | null"
      }
    ],
    "summary": {
      "supported_count": <integer>,
      "contradicted_count": <integer>,
      "unverifiable_count": <integer>,
      "human_review_count": <integer>,
      "overall_reliability_score": <float 0.0-1.0>
    }
  }
}

## CONSTRAINTS

- Do not fabricate evidence. If the Reference Document does not address a claim, mark it UNVERIFIABLE.
- Do not omit claims. Extract every factual assertion, including implicit ones.
- Do not paraphrase claim_text. Use the exact wording from the Target Text.
- If the Target Text contains no factual claims, return an empty claims array with total_claims_extracted set to 0.
- If the Reference Document is empty or missing, mark all claims as UNVERIFIABLE with confidence 0.0.

Adaptation guidance: Replace [INPUT_TEXT] with the content you want to audit, and [REFERENCE_TEXT] with your ground-truth document. Set [CONFIDENCE_THRESHOLD] to a value like 0.7 for moderate sensitivity or 0.9 for high precision. [HIGH_RISK_DOMAINS] should be a comma-separated list such as medical advice, legal interpretation, financial projections. The [TEXT_ID] and [REF_ID] fields are for your own traceability. If your model does not reliably produce valid JSON, pair this prompt with a structured output API parameter or a post-generation JSON repair step. For regulated domains, always route flagged claims to a human reviewer before any downstream action.

The Reference-Backed Claim Verification prompt is designed to operate as a structured audit step inside a content pipeline, not as a standalone chat interaction. The typical integration point is after a model generates content and before that content reaches a human reviewer or end user. The harness must supply the prompt with a complete [CLAIMS_LIST] extracted from the generated output, a [REFERENCE_TEXT] containing the authoritative source material, and a [VERIFICATION_SCHEMA] that defines the expected JSON output shape. The prompt returns a claim-level audit with confidence scores, evidence spans, and flags for human review. This output should be treated as a machine-readable signal, not a final decision.

Validation and retry logic is critical because the prompt produces structured JSON that downstream systems consume. Implement a JSON schema validator that checks for required fields (claim_id, verdict, confidence, evidence_span, explanation, requires_human_review) and correct types before accepting the response. If validation fails, retry up to two times with the same inputs and an added instruction to fix the specific schema error. After two retries, escalate the entire batch to a human review queue. For high-throughput pipelines, consider batching claims into groups of 5-10 to keep individual prompt calls fast and retry scope small. Log every verification attempt with the model name, prompt version, input claims, raw output, validation result, and retry count for auditability.

Model choice and tool integration depend on the risk level of the content domain. For low-risk content (e.g., internal documentation drafts), a fast model like GPT-4o-mini or Claude Haiku with temperature set to 0 is sufficient. For regulated domains (e.g., clinical summaries, financial reports, legal content), use a more capable model like GPT-4o or Claude Sonnet, set temperature=0, and route all requires_human_review: true claims to a dedicated review interface. If your pipeline already uses retrieval-augmented generation, the [REFERENCE_TEXT] should be the same retrieved context that the generation model used, ensuring consistency between what the model saw and what gets verified. Do not use this prompt as a substitute for source retrieval—it verifies claims against provided evidence, it does not search for evidence independently.

Human review integration is the most important architectural decision. Claims flagged with requires_human_review: true and confidence below 0.7 should appear in a review queue with the original claim, the reference evidence span, and the model's explanation pre-filled. The reviewer should be able to confirm, overturn, or amend the verdict with a single click, and that human decision should be logged as ground truth for future eval calibration. Over time, track the rate of human overrides by claim type and evidence source to identify systematic weaknesses in either the generation model or the verification prompt. If the override rate exceeds 15% for any category, that's a signal to update the verification prompt, improve the reference material, or add few-shot examples for that claim type.

Evaluation and monitoring should run continuously, not just at deployment time. Maintain a golden dataset of 50-100 claim-reference pairs with known-correct verdicts, and run the verification prompt against this dataset weekly or on every prompt version change. Measure precision, recall, and F1 for each verdict category (SUPPORTED, UNSUPPORTED, CONTRADICTED). Track confidence calibration by comparing predicted confidence against actual correctness—a well-calibrated prompt should have high confidence for correct verdicts and low confidence for errors. Set an alert if F1 drops below 0.85 or if the human override rate spikes after a model or prompt update. This prompt is a product component, and it needs the same operational rigor as any other production service.