Claim Extraction and Source Verification Prompt

This prompt is designed for accuracy-critical workflows where every factual assertion in a model's output must be traced back to a source document or explicitly flagged as unsupported. Use it as a post-generation verification step in RAG pipelines, document intelligence systems, compliance review tools, and any product where fabricated claims create unacceptable risk. The ideal user is an AI engineer or product team building a system that requires auditable, evidence-backed outputs, such as a legal document analyzer, a medical summarization tool, or a financial report generator. The core job-to-be-done is transforming a generated text and its source context into a structured, machine-readable verification report that can trigger downstream actions like regeneration, human review, or confidence scoring.

The prompt requires two critical inputs to function: the [GENERATED_TEXT] to be verified and the [SOURCE_CONTEXT] from which all supporting evidence must be drawn. Without a well-defined source context, the prompt cannot perform its core function and will correctly flag all claims as unsupported. It is not a general-purpose fact-checker for claims about the world outside the provided context, and it does not replace human review in regulated domains. For high-stakes applications in healthcare, law, or finance, the structured output from this prompt should be treated as a triage and routing mechanism, directing a human reviewer's attention to the most critical unsupported claims rather than serving as a final arbiter of truth.

Do not use this prompt when the source context is unavailable, when the generated text is purely creative or subjective, or when the cost and latency of a second verification pass are unacceptable for the use case. It is also inappropriate for real-time chat applications where a structured verification report would disrupt the user experience. Instead, consider using a constrained generation prompt like the 'Context-Only Answer Constraint Prompt Template' to prevent hallucination upfront. For production deployments, always pair this prompt with a validation harness that checks the output schema, logs verification reports for audit trails, and implements a retry budget with escalation to human review when unsupported claims persist.

This template is the core instruction set for the Claim Extraction and Source Verification workflow. It instructs the model to decompose a generated response into atomic, verifiable claims, then cross-reference each claim against the provided source context. The output is a structured verification report that downstream systems can parse, log, and act on. Replace every square-bracket placeholder with your actual data before sending this to the model. The prompt is designed to be stateless and idempotent—running it multiple times on the same input should produce a structurally identical report.

textCopy
You are a factual accuracy auditor. Your task is to extract every discrete factual claim from the [MODEL_OUTPUT] below, verify each claim against the [SOURCE_CONTEXT], and produce a structured verification report.

## INPUT
[MODEL_OUTPUT]:
"""
[PASTE_MODEL_OUTPUT_HERE]
"""

[SOURCE_CONTEXT]:
"""
[PASTE_SOURCE_CONTEXT_HERE]
"""

## INSTRUCTIONS
1. Decompose the [MODEL_OUTPUT] into atomic, self-contained factual claims. Each claim must be a single verifiable statement. Do not group multiple facts into one claim.
2. For each claim, search the [SOURCE_CONTEXT] for direct supporting evidence. A claim is SUPPORTED only if the evidence explicitly states the fact. Inferences and implications are not sufficient.
3. Assign a verification status to each claim:
   - SUPPORTED: The claim is directly stated in the [SOURCE_CONTEXT]. Include the exact quote and its location.
   - UNSUPPORTED: The claim cannot be verified from the [SOURCE_CONTEXT]. This includes plausible inferences, background knowledge, and statements that feel true but lack explicit evidence.
   - CONTRADICTED: The claim directly conflicts with evidence in the [SOURCE_CONTEXT].
4. If no factual claims are present in the [MODEL_OUTPUT], return an empty claims array.
5. Do not evaluate the quality, style, or usefulness of the [MODEL_OUTPUT]. Only assess factual grounding.

## OUTPUT_SCHEMA
Return a JSON object with this exact structure:
{
  "verification_report": {
    "total_claims": <integer>,
    "supported_claims": <integer>,
    "unsupported_claims": <integer>,
    "contradicted_claims": <integer>,
    "claims": [
      {
        "claim_id": <integer>,
        "claim_text": "<string>",
        "status": "SUPPORTED | UNSUPPORTED | CONTRADICTED",
        "source_quote": "<string or null if unsupported>",
        "source_location": "<string or null if unsupported>",
        "contradicting_evidence": "<string or null if not contradicted>",
        "notes": "<string: brief explanation of the verification decision>"
      }
    ]
  }
}

## CONSTRAINTS
- Do not fabricate source quotes. If no evidence exists, set source_quote and source_location to null.
- Do not skip claims that seem obvious or common knowledge. Every factual statement must be checked.
- If the [SOURCE_CONTEXT] is empty or irrelevant, mark all claims as UNSUPPORTED.
- Return ONLY the JSON object. No preamble, no commentary, no markdown fences.

Adaptation guidance: The [MODEL_OUTPUT] placeholder should receive the complete text you want to audit—do not truncate or summarize it beforehand. The [SOURCE_CONTEXT] should contain the full set of retrieved passages, documents, or ground-truth data the model was supposed to use. If your system uses multiple source documents, concatenate them with clear document boundary markers (e.g., [DOC_ID: abc123]) so the verification report can reference specific locations. For high-stakes domains like healthcare or finance, add a [RISK_LEVEL] parameter that gates whether UNSUPPORTED claims trigger an automatic regeneration or a human review queue. The output schema is intentionally flat to simplify downstream parsing; if you need nested claim grouping, extend the schema but preserve the claim_id and status fields as the minimum contract.

This prompt is designed to be called as a post-generation verification step, not a user-facing endpoint. In a typical harness, you first generate a model response (e.g., a RAG answer, a summary, or an analysis), then pass that output along with the original source context into this prompt. The prompt extracts discrete factual claims, maps each to source evidence, and flags unsupported assertions. The output is a structured verification report that your application can parse and act on programmatically.

Integration pattern: Wrap this prompt in a verification service that accepts (generated_text, source_context, claim_schema) and returns a structured payload. Validate the output against a strict JSON schema before processing. Key fields to enforce: claim_id (string), claim_text (string), source_reference (string or null), verification_status (enum: supported, unsupported, partially_supported, contradicted), evidence_excerpt (string or null), and confidence (float 0-1). If the model returns malformed JSON, use a repair prompt from the Output Repair pillar rather than retrying the verification prompt blindly. Log every verification result with a trace ID that links back to the original generation request for debugging.

Retry and escalation logic: If the verification report indicates unsupported claims above a configurable threshold (e.g., more than 10% of extracted claims are unsupported), trigger a grounded answer regeneration prompt from the Hallucination and Factual Error Self-Correction group. Pass the flagged claims and source context into that regeneration prompt. Track retry counts per request. After three regeneration attempts, if unsupported claims persist, escalate to a human review queue with the full verification report and regeneration history. Never loop indefinitely—set a hard retry budget and log exhaustion events for monitoring.

Model choice and latency: This prompt works best with models that have strong instruction-following and structured output capabilities. For high-throughput pipelines, consider routing to a smaller, faster model for initial claim extraction and a larger model only when the smaller model's confidence scores fall below a threshold. Cache verification results keyed by a hash of (generated_text, source_context) to avoid redundant verification of identical outputs. If latency is critical, run verification asynchronously and surface results in a follow-up event rather than blocking the user response.

Observability and evals: Instrument the verification pipeline with metrics: claim extraction rate, supported vs. unsupported ratio, average confidence, JSON parse failure rate, and retry escalation rate. Build an eval set of known hallucinated outputs with ground-truth verification labels. Run this eval set on every prompt change and model migration. Compare verification reports across versions to detect regression in detection accuracy. Store verification reports alongside generation logs so you can trace any user-reported hallucination back to whether the verifier caught it, missed it, or was never invoked.