Disputed Fact Extraction Prompt Template

This prompt is designed for fact-checking and verification pipelines that must surface disagreements across a set of sources. It extracts factual claims that are actively disputed, identifies which sources support and refute each claim, and classifies the nature of the dispute. Use this when you need structured, machine-readable conflict data before generating an answer or briefing. This prompt belongs in the evidence conflict detection stage, after retrieval but before synthesis or user-facing presentation. It is not a general summarization tool and should not be used when sources are expected to be in complete agreement.

The ideal user is an AI pipeline engineer or product developer building a research assistant, decision-support system, or verification workflow. The prompt requires a set of retrieved passages or documents as input, along with a target claim or question to anchor the dispute detection. It expects the model to output a structured list of disputed facts, each with supporting and refuting source citations and a dispute classification. Do not use this prompt on single-source inputs, on passages that all originate from the same underlying report, or in contexts where the goal is to produce a unified answer rather than a conflict map. Using it on uncontested background information will produce false positives or empty results.

Before wiring this into production, define clear evaluation criteria: precision of extracted disputes (avoiding false positives on paraphrased agreement), recall of genuine contradictions, and accuracy of source attribution. Run the prompt against a golden dataset of known conflicts and uncontested passages to calibrate thresholds. If the downstream system will present conflicts to end users, add a human review step for high-severity or high-impact disputes before they reach the UI. The output of this prompt feeds into synthesis, resolution, or presentation prompts—never directly to users without validation.

This prompt template is designed to be pasted directly into your system instructions for a fact-checking or verification pipeline. It instructs the model to extract only claims where sources actively disagree, ignoring uncontested background information. The template uses square-bracket placeholders that you must replace with your specific data before sending the request to the model. The output is structured JSON suitable for downstream processing, audit logging, or human review queues.

textCopy
You are a disputed fact extraction system. Your task is to analyze a set of source passages and identify factual claims where sources disagree. You must distinguish genuine disputes from uncontested background information, differences in scope, or paraphrased agreement.

## INPUT

**Claim Context:** [CLAIM_OR_QUESTION]

**Source Passages:**
[SOURCE_PASSAGES]

## OUTPUT SCHEMA

Return a JSON object with the following structure:

{
  "disputed_claims": [
    {
      "claim_id": "string",
      "claim_text": "string (the specific factual statement in dispute)",
      "supporting_sources": [
        {
          "source_id": "string",
          "source_excerpt": "string (the exact text supporting the claim)",
          "support_strength": "direct | indirect | weak"
        }
      ],
      "refuting_sources": [
        {
          "source_id": "string",
          "source_excerpt": "string (the exact text refuting the claim)",
          "refutation_strength": "direct | indirect | weak"
        }
      ],
      "dispute_type": "factual_contradiction | methodological_difference | interpretation_divergence | scope_mismatch | temporal_change",
      "confidence": "high | medium | low",
      "resolution_notes": "string (brief assessment of whether the dispute is resolvable with available evidence)"
    }
  ],
  "uncontested_claims": [
    {
      "claim_text": "string",
      "supporting_sources": ["source_id"],
      "confidence": "high | medium | low"
    }
  ],
  "extraction_metadata": {
    "total_sources_analyzed": "number",
    "total_disputes_found": "number",
    "limitations_notes": "string (any caveats about the extraction quality)"
  }
}

## CONSTRAINTS

1. Only extract claims where at least two sources actively disagree. Do not flag differences in emphasis, wording, or level of detail as disputes.
2. For each disputed claim, you must provide exact excerpts from both supporting and refuting sources. Never paraphrase source content in the excerpts.
3. If sources appear to conflict but actually discuss different time periods, populations, or conditions, classify the dispute_type as "scope_mismatch" or "temporal_change" rather than "factual_contradiction."
4. If multiple sources agree on a claim but cite the same original source, do not treat this as independent corroboration. Note this in resolution_notes.
5. If no disputes are found, return an empty disputed_claims array and explain why in extraction_metadata.limitations_notes.
6. Set confidence to "low" when the dispute relies on inference rather than explicit contradiction, or when source reliability is questionable.

## EXAMPLES

**Example 1: Direct Factual Contradiction**

Source A: "The company reported revenue of $4.2 billion in Q3 2024."
Source B: "The company's Q3 2024 revenue was $3.8 billion."

Output:
{
  "claim_text": "The company's Q3 2024 revenue was $4.2 billion",
  "dispute_type": "factual_contradiction",
  "confidence": "high"
}

**Example 2: Temporal Change (Not a Dispute)**

Source A: "The policy was in effect from January 2023."
Source B: "The policy took effect in March 2023 after amendments."

Output:
{
  "claim_text": "The policy effective date",
  "dispute_type": "temporal_change",
  "confidence": "medium",
  "resolution_notes": "Source A may reference the original announcement date; Source B references the amended effective date. Not a direct contradiction."
}

**Example 3: Paraphrased Agreement (Not a Dispute)**

Source A: "The treatment reduced symptoms in 67% of patients."
Source B: "Approximately two-thirds of patients experienced symptom reduction."

Output: Do not extract as disputed. These sources agree.

## RISK LEVEL

[RISK_LEVEL]

If RISK_LEVEL is "high": Flag all disputed claims for human review before any downstream use. Include a note in extraction_metadata indicating that human verification is required.

To adapt this template, replace the square-bracket placeholders with your actual data. [CLAIM_OR_QUESTION] should contain the specific claim or question being investigated. [SOURCE_PASSAGES] should contain the full text of each source passage, each clearly labeled with a source_id that matches the identifiers used in the output schema. [RISK_LEVEL] should be set to "high" for regulated domains, customer-facing claims, or decisions with significant consequences; set to "medium" or "low" for internal research or exploratory analysis where human review is still recommended but not mandatory for every dispute. After adapting the template, test it against a golden dataset of known disputes and non-disputes to verify that the model correctly distinguishes genuine conflicts from paraphrased agreement and scope differences.

The Disputed Fact Extraction prompt is not a standalone widget; it is a processing stage in a fact-checking or verification pipeline. In production, you will typically call this prompt after retrieving a set of source documents or passages relevant to a target claim or topic. The prompt expects a structured input containing the claim under review and the full text of each source, and it returns a structured output of disputed facts, supporting and refuting sources, and dispute classifications. The harness around this prompt is responsible for input assembly, output validation, retry logic, logging, and escalation when the model cannot resolve a dispute with sufficient confidence.

Begin by constructing the input payload carefully. The [CLAIM] placeholder should receive the exact claim text being verified. The [SOURCES] placeholder should receive a JSON array of source objects, each with source_id, source_text, and optional source_metadata fields (e.g., publication date, author, domain). If your retrieval system returns more sources than the model's context window can hold, implement a pre-ranking step using a lighter relevance model or a keyword overlap heuristic to select the top-N most relevant sources. After the model returns its response, validate the output against a strict schema: each disputed fact must have a non-empty claim field, at least one supporting_sources or refuting_sources entry, and a dispute_type from your allowed enum (e.g., factual_contradiction, interpretation_divergence, scope_mismatch, temporal_change). Reject any output that contains claims not present in the input sources or that classifies a fact as disputed when all sources agree. A second validation pass should check that every cited source_id actually exists in the input payload to prevent hallucinated citations.

Implement a retry recovery loop for validation failures. If the output fails schema validation, feed the validation error messages back into the prompt as additional [CONSTRAINTS] and request a corrected output. Limit retries to two attempts before escalating to a human review queue. For high-stakes domains such as healthcare, legal, or finance, always route outputs with dispute_type values of factual_contradiction or interpretation_divergence to a human reviewer before the results are surfaced to end users. Log every prompt call with the input claim, source IDs, raw model output, validation results, retry count, and final disposition. This audit trail is essential for debugging false positives (e.g., the model flagging paraphrased agreement as a dispute) and false negatives (e.g., missing contradictions that require inference across sources).

Model choice matters for this workflow. The prompt requires strong instruction-following, structured output generation, and the ability to compare claims across multiple documents. Prefer models with native JSON mode or structured output support to reduce parsing failures. If you are using a model without guaranteed schema adherence, add a post-processing repair step that attempts to fix common formatting errors (e.g., unescaped quotes, trailing commas) before falling back to a retry. For cost-sensitive pipelines, consider routing simpler cases—such as single-source verification or claims with obvious consensus—to a smaller, faster model, and reserving the full multi-source dispute extraction prompt for cases where an initial relevance or contradiction signal suggests conflict is likely.

Finally, integrate this prompt into a broader evaluation framework. Maintain a golden dataset of claims with known disputes and known consensus cases. Run the prompt against this dataset on every prompt change, model upgrade, or pipeline modification. Measure precision (what fraction of flagged disputes are genuine conflicts) and recall (what fraction of known disputes are detected). Track false positives from paraphrased agreement and false negatives from implicit contradictions. If precision drops below an acceptable threshold, tune the prompt's instructions for distinguishing genuine conflict from stylistic or scope differences. If recall drops, consider adding a pre-processing step that rewrites source passages into a common claim structure before comparison.