Attributed Statement Verification Prompt

This prompt is the decision engine for a verification pipeline, not a standalone fact-checker. Its job is to confirm or refute whether a specific public figure made a particular statement, but it can only do so by reasoning over evidence you provide. The ideal user is an AI engineer or backend developer who has already built a retrieval step—web search, internal document corpus, or news archive—and needs a reliable, schema-constrained verification decision. The prompt accepts an attributed statement and a subject identifier, then expects the retrieved evidence to be injected alongside it. Without that evidence, the model is explicitly instructed to abstain rather than rely on its parametric knowledge, preventing hallucinated verifications that erode trust in your pipeline.

You should use this prompt when your application requires a structured, auditable verification output that includes a verification status, source provenance, a calibrated confidence score, and any contradictory evidence found. It is designed for high-stakes editorial, compliance, or research workflows where the cost of a false positive (verifying a false statement) or false negative (flagging a true statement) is significant. The output schema is strict, making it suitable for direct ingestion into a database, a CMS plugin, or a human review queue. Do not use this prompt for open-ended research questions, subjective opinion analysis, or verifying claims that lack a clear public-figure attribution. It is also not a replacement for a retrieval system; if your pipeline cannot supply relevant text snippets, this prompt will simply return an ABSTAIN status, which is the correct and safe behavior.

Before wiring this into production, understand its primary failure mode: evidence quality. The model's verification is only as good as the retrieval step that feeds it. If your search returns irrelevant, outdated, or satirical sources, the prompt may produce a false VERIFIED or REFUTED status with high confidence. To mitigate this, always pair this prompt with a robust retrieval and source-authority scoring layer. The next section provides the copy-ready template. After adapting it, proceed to the implementation harness section to learn how to add validation, retries, and human review gates before shipping.

This template is designed to be pasted directly into your system instructions for an LLM that has access to web-search and document-retrieval tools. It forces the model to act as a verification engine, not a conversational assistant. The prompt requires the model to ground every determination in retrieved evidence, produce a structured JSON output, and abstain when evidence is insufficient. Before using this template, ensure your retrieval pipeline can supply live search results or document chunks into the [RETRIEVED_EVIDENCE] placeholder. The prompt is opinionated about output structure because downstream systems—dashboards, review queues, audit logs—need predictable fields, not prose.

textCopy
You are a statement verification engine. Your sole task is to determine whether an attributed statement was actually said or written by the named person, using only the provided evidence. You do not answer questions, provide opinions, or generate content outside the verification task.

## INPUT
Attributed Statement: [STATEMENT]
Attributed Person: [PERSON_NAME]
Attribution Context (date, venue, publication, surrounding claims): [ATTRIBUTION_CONTEXT]
Retrieved Evidence (search results, document excerpts, transcripts): [RETRIEVED_EVIDENCE]

## OUTPUT SCHEMA
Return ONLY valid JSON matching this schema:
{
  "verification_status": "verified" | "contradicted" | "unverifiable" | "misattributed",
  "confidence_score": 0.0-1.0,
  "confidence_rationale": "string explaining calibration",
  "matching_evidence": [
    {
      "source_title": "string",
      "source_url": "string or null",
      "excerpt": "relevant passage",
      "match_type": "exact_match" | "paraphrase_match" | "contextual_match",
      "date_of_source": "ISO date or null"
    }
  ],
  "contradictory_evidence": [
    {
      "source_title": "string",
      "source_url": "string or null",
      "excerpt": "contradicting passage",
      "contradiction_type": "direct_denial" | "contextual_conflict" | "temporal_impossibility"
    }
  ],
  "missing_context_warnings": ["string flags for context-stripping risks"],
  "abstention_reason": "string or null (required if status is unverifiable)"
}

## VERIFICATION RULES
1. **Evidence-first**: Every status determination must cite specific evidence from the `[RETRIEVED_EVIDENCE]` field. Do not rely on training data knowledge.
2. **Exact match requires verbatim text**: Only mark `match_type` as `exact_match` if the retrieved source contains the identical wording. Minor punctuation differences still qualify.
3. **Paraphrase match requires meaning preservation**: A `paraphrase_match` means the core claim is substantively the same, even if wording differs. Flag any semantic drift in `missing_context_warnings`.
4. **Contradiction requires direct conflict**: Only populate `contradictory_evidence` when a source explicitly states something incompatible with the attributed statement. Silence or absence is not contradiction.
5. **Unverifiable is a valid outcome**: If evidence is insufficient, conflicting, or too low-quality to decide, set `verification_status` to `unverifiable` and explain in `abstention_reason`. Do not guess.
6. **Confidence calibration**: Score 0.9+ only for exact matches from authoritative primary sources (official transcripts, verified social accounts, reputable publications with bylines). Score 0.7-0.89 for strong paraphrase matches from reliable secondary sources. Score 0.5-0.69 for partial matches or single-source corroboration. Score below 0.5 for weak evidence, anonymous sources, or significant context gaps. Always explain your calibration in `confidence_rationale`.
7. **Misattribution handling**: If evidence shows the statement was made by someone else or fabricated, set status to `misattributed` and include the correct attribution in `contradictory_evidence`.
8. **Context-stripping detection**: If the attributed statement omits qualifiers present in the original source (e.g., "I think," "possibly," "in some cases"), add a warning to `missing_context_warnings` even if the core words match.

## TOOL USE
You have access to web_search and document_retrieval tools. Use them to supplement `[RETRIEVED_EVIDENCE]` if the provided evidence is insufficient. However, always cite the specific source you retrieve. Do not blend retrieved evidence with training data assumptions.

## CONSTRAINTS
- Do not output markdown, explanations, or conversational text outside the JSON.
- Do not invent sources, URLs, or excerpts.
- Do not treat the absence of contradictory evidence as confirmation.
- If the attributed person is a public figure with known impersonation accounts, prioritize verified sources.
- For statements attributed to deceased persons, flag temporal impossibility if the statement references events after their death.

Adaptation guidance: Replace the square-bracket placeholders with live data from your retrieval pipeline. The [RETRIEVED_EVIDENCE] field should contain pre-fetched search results or document chunks formatted as structured text with source metadata. If your application already has a tool-use loop, you can remove the [RETRIEVED_EVIDENCE] placeholder and let the model call search tools directly—but always keep the citation requirement. For high-stakes domains (legal evidence, journalism, compliance), add a [RISK_LEVEL] parameter that adjusts the confidence threshold for auto-verification and routes low-confidence results to human review. The output schema is intentionally flat to simplify parsing; if your pipeline needs nested structures (e.g., per-claim verification within a multi-claim statement), extend the schema but preserve the verification_status and confidence_score fields as the primary routing signals.

The Attributed Statement Verification Prompt is designed as a single step in a larger verification pipeline, not as a standalone chatbot interaction. The prompt expects a specific input shape—an attributed statement and a set of retrieved source documents—and returns a structured verification verdict. In production, you will wrap this prompt in an application harness that handles pre-processing (claim extraction, source retrieval), post-processing (schema validation, confidence thresholding), and orchestration (retries, fallbacks, human review routing). The harness is responsible for ensuring the model receives clean, bounded inputs and that its output is machine-verifiable before any downstream action is taken.

Start by defining a strict input contract. The [ATTRIBUTED_STATEMENT] placeholder should receive a single, atomic claim with an explicit attribution target (e.g., 'CEO Jane Smith stated that Q3 revenue grew 12%'). The [RETRIEVED_SOURCES] placeholder expects a pre-assembled context block containing the top-k documents from your retrieval system, each with a source ID, title, date, and snippet. Do not pass raw search engine results directly into the prompt. Instead, use a retrieval pipeline—keyword search, vector search, or hybrid—to fetch candidate sources, then apply a relevance filter and deduplication step before assembly. For high-stakes verification, include a source authority score from a separate assessment step so the model can weight evidence appropriately. The [VERIFICATION_DATE] and [DOMAIN] fields help the model calibrate recency expectations and terminology handling.

After the model returns a response, validate the output against the expected JSON schema before any downstream consumption. The prompt specifies fields like verification_status, confidence_score, source_provenance, and contradictory_evidence. Your harness should check that verification_status is one of the allowed enum values (verified_true, verified_false, unverifiable, disputed), that confidence_score is a float between 0.0 and 1.0, and that source_provenance contains valid source IDs that map back to the input sources. If schema validation fails, implement a retry loop: re-invoke the model with the same inputs plus a repair instruction that includes the specific validation error. Limit retries to two attempts before escalating to a human reviewer or logging the failure for offline analysis. For unverifiable or disputed outputs, route the verification packet—including the original statement, retrieved sources, and model output—to a human review queue with a structured context summary.

Model choice matters for this workflow. Use a model with strong instruction-following and structured output capabilities. At the time of writing, GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro all perform well on attributed verification tasks when given clean source context. For cost-sensitive or high-throughput pipelines, consider routing simple, high-confidence cases to a smaller model (e.g., GPT-4o-mini or Claude 3 Haiku) and escalating ambiguous or low-confidence cases to a larger model. Implement a confidence threshold in your harness: if confidence_score falls below 0.7, automatically escalate to a stronger model or human review. Log every verification decision with the full input, output, validation result, and routing decision for auditability and prompt debugging.

Tool integration is essential for this prompt to function at scale. The prompt itself does not call tools; your harness must handle retrieval before invocation. Connect the harness to a web search API (e.g., Bing, Brave, or Tavily) for real-time claims, or to an internal document store (e.g., a vector database like Pinecone or Weaviate) for claims against known corpora. For claims about public figures, consider integrating with structured knowledge sources like Wikidata or Wikipedia APIs to cross-reference biographical facts. The harness should also support a 'source freshness' check: if the most recent retrieved source is older than a configurable threshold relative to the claim date, flag the verification as potentially stale and adjust the confidence score downward or route to human review.

Before deploying, build a regression test suite with at least 50 labeled examples covering true statements, false statements, unverifiable statements, and edge cases like statements with partial evidence or conflicting sources. For each test case, store the expected verification_status, an acceptable confidence_score range, and the required source IDs. Run this suite on every prompt or model change. Additionally, implement an LLM-as-judge eval step that compares the harness output against ground truth using a rubric focused on status accuracy, source attribution correctness, and contradiction detection. Monitor production performance by tracking the rate of schema validation failures, human review escalations, and user-reported errors. The most common failure mode is the model hallucinating source details when the retrieved context is insufficient—your harness must treat missing source IDs in the output as a critical error and never surface unverifiable provenance to end users.