Multi-Source Claim Verification Prompt

This prompt is designed for a single, high-stakes task: assessing whether a specific claim is supported, contradicted, or unaddressed by a provided set of evidence passages. It is a calibration instrument, not a conversational agent. The ideal user is an AI engineer or product developer building a fact-checking pipeline, a RAG system with a verification guardrail, or an internal tool that must flag unsubstantiated statements before they reach a user. The required context is a pre-retrieved set of evidence passages and a single, atomic claim to verify. This prompt belongs in the narrow window after retrieval and before any answer generation, publication, or decision record is created. It is not a substitute for a retrieval query generator, a general question-answering prompt, or a human editorial review process in domains where safety or compliance are paramount.

You should use this prompt when the cost of an unverified claim is high—for example, in financial analysis, medical information systems, legal research, or any product where user trust depends on demonstrated accuracy. The prompt is structured to produce a machine-readable verification verdict with per-source support levels, an aggregate confidence score, and a narrative explanation of agreement or disagreement patterns. This structured output is designed to be consumed by downstream logic: a high-confidence 'contradicted' verdict can block a response, while an 'unaddressed' verdict can trigger a new retrieval loop or a human review queue. The prompt assumes that the evidence passages are already retrieved and relevant; it does not perform its own retrieval or assess the quality of the retrieval step. If your passages are noisy, off-topic, or insufficient, the prompt will faithfully report that the claim is unaddressed, which is the correct and safe behavior.

Do not use this prompt as a general-purpose fact-checker that searches the web or relies on the model's internal knowledge. It is explicitly designed to be evidence-grounded, meaning it must only reason over the provided passages. Using it without supplying evidence will produce unreliable results. Do not use it to verify multiple claims in a single call; the atomic design ensures that each claim gets a focused assessment, and batching claims will degrade the per-claim analysis. Finally, do not treat the output as a final, publishable verdict in high-stakes domains without human review. The prompt is a powerful filter and triage tool, but it is not a replacement for editorial judgment, domain expertise, or legal and medical review. The next step after reading this section should be to review the prompt template and understand the required input variables, then study the implementation harness to see how validation, retries, and human-in-the-loop escalation are wired around the prompt.

This template is the core instruction set for a multi-source claim verification workflow. It accepts a claim to verify, a set of source passages with metadata, and optional calibration examples that anchor the model's confidence scale. The prompt is designed to produce structured JSON output suitable for downstream automated processing, human review queues, or audit logging. Replace every square-bracket placeholder with your data before sending the prompt to the model. The template includes explicit instructions for handling source disagreement, insufficient evidence, and edge cases where the claim is partially supported.

textCopy
You are a claim verification system. Your task is to assess whether a claim is supported by the provided source passages.

## INPUT
Claim to verify: [CLAIM]

Source passages:
[SOURCE_PASSAGES]

Calibration examples:
[CALIBRATION_EXAMPLES]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "verdict": "SUPPORTED" | "PARTIALLY_SUPPORTED" | "UNSUPPORTED" | "CONTRADICTED" | "INSUFFICIENT_EVIDENCE",
  "aggregate_confidence": 0.0-1.0,
  "confidence_rationale": "string explaining what drives the confidence level",
  "per_source_assessment": [
    {
      "source_id": "string matching the source identifier provided",
      "support_level": "FULLY_SUPPORTS" | "PARTIALLY_SUPPORTS" | "NEUTRAL" | "PARTIALLY_CONTRADICTS" | "FULLY_CONTRADICTS" | "NO_RELEVANT_INFORMATION",
      "explanation": "string explaining how this source relates to the claim",
      "relevant_excerpts": ["string excerpts from the source that inform this assessment"]
    }
  ],
  "agreement_pattern": "UNANIMOUS_SUPPORT" | "MAJORITY_SUPPORT" | "MIXED" | "MAJORITY_CONTRADICTION" | "UNANIMOUS_CONTRADICTION" | "INSUFFICIENT_DATA",
  "agreement_explanation": "string describing the pattern of agreement or disagreement across sources",
  "limitations": ["list of factors that limit the reliability of this verification"],
  "requires_human_review": true | false
}

## CONSTRAINTS
- Base your assessment ONLY on the provided source passages. Do not use external knowledge.
- If no source contains relevant information, use verdict INSUFFICIENT_EVIDENCE.
- If sources directly contradict each other, reflect this in the agreement_pattern and per_source_assessment. Do not fabricate consensus.
- Use the calibration examples to anchor your confidence scale: a confidence of 0.9 should mean the same level of certainty as in the provided high-confidence calibration example.
- Mark requires_human_review as true if: sources conflict, confidence is below 0.7, the claim involves [HIGH_RISK_DOMAINS], or evidence is insufficient for a definitive verdict.
- For partial support, explain exactly which parts of the claim are supported and which are not.
- Extract relevant_excerpts verbatim from the source passages. Do not paraphrase or summarize in this field.

## RISK LEVEL
[RISK_LEVEL]

## INSTRUCTIONS
1. Read the claim carefully. Identify all factual assertions within it.
2. For each source passage, determine whether it contains information relevant to each assertion.
3. Assess the support level for each source independently before determining the aggregate verdict.
4. If calibration examples are provided, use them to calibrate your confidence scores.
5. Return ONLY the JSON object. No additional text before or after.

Adaptation guidance: The [CLAIM] placeholder accepts a single statement or a structured object with sub-claims. For multi-assertion claims, consider splitting into separate verification passes. The [SOURCE_PASSAGES] placeholder should contain source objects with at minimum an id and text field; add authority_score, recency, or source_type fields if your pipeline pre-scores sources. The [CALIBRATION_EXAMPLES] placeholder is optional but strongly recommended—provide at least one known-true claim with its sources and one known-false claim to anchor the model's confidence scale. The [HIGH_RISK_DOMAINS] placeholder should be replaced with a comma-separated list of domains that trigger mandatory human review, such as "medical claims, legal interpretations, financial projections." The [RISK_LEVEL] placeholder accepts "LOW", "MEDIUM", or "HIGH" and should be set based on your application's tolerance for verification errors. For production pipelines, always validate the output JSON against the schema before allowing the result to reach downstream consumers or decision points.

The Multi-Source Claim Verification Prompt is designed to operate as a stateless verification step within a larger fact-checking or RAG pipeline. It expects a claim and a set of pre-retrieved source passages as input, and it returns a structured verdict with per-source support levels, aggregate confidence, and an explanation of agreement or disagreement patterns. This prompt does not perform retrieval itself—it assumes the calling application has already gathered candidate evidence and formatted it into the [SOURCES] placeholder. In production, you will typically call this prompt after a retrieval step and before presenting a verified answer to the user or writing results to an audit log.

Integration pattern: Wrap the prompt in a function that accepts a claim string and a sources array of objects with source_id, text, and optional credibility_score fields. Construct the prompt by injecting these into the [CLAIM] and [SOURCES] placeholders. The model should return a JSON object matching the [OUTPUT_SCHEMA] you define—typically including verdict (supported, partially_supported, contradicted, insufficient_evidence), per_source_support (an array mapping each source_id to a support_level and rationale), aggregate_confidence (a float 0.0–1.0 or categorical label), and explanation (a summary of agreement or disagreement patterns). Validation layer: Before accepting the output, validate that every source_id in the response matches an input source, that aggregate_confidence falls within the expected range, and that the verdict is consistent with the per-source support levels. If validation fails, retry once with the error message appended to the prompt context. If retry also fails, escalate to a human review queue with the claim, sources, and partial output attached.

Model choice and latency: For high-throughput pipelines, use a fast instruction-tuned model (e.g., GPT-4o-mini, Claude 3.5 Haiku) with strict JSON mode enabled. For high-stakes verification where nuanced disagreement detection matters, route to a more capable model (e.g., GPT-4o, Claude 3.5 Sonnet) and accept the latency trade-off. Logging and audit: Log every verification call with the claim, source IDs, model output, validation result, and final disposition. This audit trail is essential for calibration analysis—you can compare model verdicts against known-true and known-false claims over time to measure drift. Calibration harness: Maintain a golden dataset of claims with ground-truth labels. Run the prompt against this dataset weekly, compute precision/recall per verdict category, and set alert thresholds for degradation. Human review triggers: Automatically escalate to human review when: (a) aggregate_confidence falls in a configured gray zone (e.g., 0.4–0.6), (b) the verdict is contradicted with high-stakes subject matter, (c) validation fails after retry, or (d) the explanation indicates sources are in direct conflict without a clear resolution path.

What to avoid: Do not use this prompt as a standalone truth oracle. It verifies claims against provided sources only—it cannot detect whether the sources themselves are wrong, outdated, or fabricated. Always pair it with source authority assessment and freshness checks upstream. Do not skip the validation layer; malformed JSON or hallucinated source IDs are common failure modes in production. Do not treat aggregate_confidence as a calibrated probability unless you have measured calibration against your own labeled data. Finally, do not expose raw verification outputs to end users without a presentation layer that translates verdicts and confidence into appropriate user-facing language and actions.