Contradictory Evidence Flagging Prompt

This prompt is designed for verification engineers and RAG pipeline builders who need to surface direct disagreements between sources rather than silently selecting one as authoritative. The core job-to-be-done is producing a structured contradiction report that quotes both sides, identifies the specific point of conflict, and suggests a resolution path—without the model taking sides. Use this when your system retrieves multiple documents that may disagree on factual claims, and downstream consumers (analysts, reviewers, or automated pipelines) need explicit conflict visibility rather than a smoothed-over consensus.

The ideal user is integrating this into a production verification pipeline where source provenance is already tracked. Required inputs include: the original claim being checked, at least two source documents with metadata (source ID, publication date, authority tier), and a defined output schema for contradiction reports. The prompt assumes evidence has already been retrieved and paired to the claim—it does not perform retrieval itself. Do not use this prompt when you have only one source, when the disagreement is purely interpretive rather than factual, or when your system's primary goal is to produce a single best answer rather than to surface uncertainty. For single-source verification, use the Claim-to-Evidence Pairing Prompt instead.

High-risk domains require additional harness controls. In legal, financial, or clinical settings, contradiction reports must never be treated as final determinations—they are triage artifacts for human review. The prompt includes a [RISK_LEVEL] parameter that adjusts the resolution suggestion language: at high risk levels, the model should refuse to recommend a resolution and instead escalate for human judgment. Always validate that both quoted passages actually appear in the cited sources before surfacing a contradiction to users. False-positive contradiction flags—where the model claims disagreement but the sources are actually consistent—are the most common production failure mode and require dedicated eval checks.

Before deploying, calibrate the prompt against a golden dataset of known contradictions and non-contradictions from your domain. Pay particular attention to near-miss cases: sources that use different terminology for the same fact, different numerical precision, or different temporal scopes. These are the cases where contradiction detection most often fails. Wire the output into a human review queue when the contradiction confidence score falls below your calibrated threshold, and log every contradiction flag with source document hashes for later audit.

This prompt template is designed to be dropped into a verification pipeline where multiple evidence sources have already been retrieved for a given claim. It forces the model to surface disagreement rather than silently selecting the most confident source. The output is a structured contradiction report that includes both sides quoted verbatim, source metadata, and a resolution suggestion. Use this when your system must demonstrate that it has considered conflicting information before producing a final answer.

textCopy
You are a contradiction detection specialist. Your task is to compare a claim against multiple evidence sources and flag any contradictory evidence.

CLAIM: [CLAIM]

EVIDENCE SOURCES:
[EVIDENCE_SOURCES]

INSTRUCTIONS:
1. For each evidence source, determine whether it supports, contradicts, or is neutral toward the claim.
2. If any sources contradict the claim or each other, produce a contradiction report.
3. If no contradictions exist, state that explicitly.
4. Quote the exact text from each source that supports your determination.
5. Include source identifiers and relevant metadata in your output.

OUTPUT SCHEMA:
{
  "claim": "[CLAIM]",
  "contradictions_found": boolean,
  "contradiction_reports": [
    {
      "source_a": {
        "id": "string",
        "position": "supports|contradicts|neutral",
        "quoted_evidence": "string",
        "metadata": {}
      },
      "source_b": {
        "id": "string",
        "position": "supports|contradicts|neutral",
        "quoted_evidence": "string",
        "metadata": {}
      },
      "contradiction_type": "direct|temporal|numerical|logical|contextual",
      "explanation": "string",
      "resolution_suggestion": "string"
    }
  ],
  "uncontradicted_sources": [
    {
      "id": "string",
      "position": "supports|neutral",
      "quoted_evidence": "string"
    }
  ]
}

CONSTRAINTS:
- Never fabricate evidence or source identifiers.
- If evidence is ambiguous, mark it as neutral rather than forcing a support/contradict classification.
- For numerical contradictions, specify the conflicting values and any tolerance windows.
- If sources disagree on a matter of interpretation rather than fact, flag it as contextual contradiction.
- Do not resolve contradictions by choosing the more authoritative source without explanation.
- If [RISK_LEVEL] is high, append a human review flag to every contradiction report.

EXAMPLES:
[EXAMPLES]

To adapt this template, replace each square-bracket placeholder with your application's runtime values. The [EVIDENCE_SOURCES] placeholder should receive pre-retrieved passages with their source IDs and metadata already attached—this prompt does not perform retrieval. The [EXAMPLES] placeholder is critical for production use: include at least two few-shot examples showing a direct contradiction and a false-positive case where sources appear to conflict but don't. For high-stakes domains, set [RISK_LEVEL] to high and ensure your harness routes flagged outputs to a human review queue. The output schema is designed for programmatic consumption; validate it against the schema before passing contradiction reports downstream.

The Contradictory Evidence Flagging Prompt is designed to be called after claim extraction and evidence retrieval have already occurred. It expects a structured input containing a specific claim and a set of evidence passages retrieved from one or more sources. The prompt's job is to compare the evidence against the claim and surface direct contradictions, not to perform retrieval or decide which source is more authoritative. This means the implementation harness must enforce a strict input contract: the claim must be a single, atomic assertion, and each evidence passage must include source metadata (document ID, publication date, authoritativeness tier) and the exact text span being compared. Without this upstream discipline, the prompt will produce noisy contradiction flags or miss genuine conflicts.

Wire this prompt as a stateless function in your verification pipeline. The call signature should accept a Claim object (with claim_id, claim_text, claim_date) and a list of Evidence objects (each with source_id, source_authority_tier, source_date, passage_text, retrieval_score). The function should validate that at least two evidence passages are present—contradiction detection requires comparison, and a single passage cannot contradict itself. Before calling the LLM, run a pre-check: if all evidence passages share the same source_id, skip the contradiction check and return a single_source_no_contradiction_check status. This prevents false-positive contradiction flags from paraphrased restatements within the same document. For model choice, use a model with strong comparative reasoning (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller or faster models for this task; contradiction detection requires careful semantic comparison that cheaper models often fail on, producing both false positives (flagging paraphrases as contradictions) and false negatives (missing genuine factual conflicts).

The output must be parsed into a structured ContradictionReport schema with fields: contradiction_detected (boolean), contradiction_type (enum: direct_factual, numerical, temporal, interpretation_conflict, none), conflicting_passages (list of passage pairs with passage_a_id, passage_b_id, conflict_description, quoted_text_a, quoted_text_b), resolution_suggestion (enum: prefer_more_recent, prefer_higher_authority, human_review_required, unresolvable), and confidence (0.0–1.0). Implement a post-processing validator that checks: (1) if contradiction_detected is true, conflicting_passages must be non-empty; (2) every passage_id referenced in the output must exist in the input; (3) quoted_text_a and quoted_text_b must be substrings of their respective input passages (with fuzzy matching to handle minor whitespace differences). If validation fails, retry once with the validation errors appended to the prompt as [PREVIOUS_OUTPUT_VALIDATION_ERRORS]. If the retry also fails validation, route to a human review queue with the raw output and validation failure details.

For high-stakes domains (healthcare, legal, financial compliance), always route contradiction_detected: true outputs with confidence < 0.85 or resolution_suggestion: human_review_required to a human reviewer. Log every contradiction flag with the full input claim, evidence set, model output, validation results, and final disposition (auto-accepted, auto-rejected, human-reviewed). This audit trail is essential for tuning the system's contradiction detection thresholds and defending decisions later. The most common production failure mode is over-flagging: the model treats differences in phrasing, scope, or granularity as contradictions. Mitigate this by maintaining a set of false-positive examples from your production logs and including them as [NEGATIVE_EXAMPLES] in the prompt template. Update these examples monthly based on human reviewer overrides.

Do not use this prompt for real-time chat or low-latency user-facing features without caching. Contradiction detection across multiple evidence passages is inherently slow (expect 2–5 seconds for a typical call). For batch verification pipelines, run contradiction checks as an asynchronous step after evidence retrieval, with a configurable timeout (recommend 10 seconds) and a dead-letter queue for timeouts. If your pipeline processes high-volume claims, consider pre-filtering: only run contradiction detection when evidence retrieval returns passages from multiple sources with conflicting sentiment or when a claim's confidence score from the evidence sufficiency step falls below a threshold. This reduces unnecessary LLM calls and keeps costs manageable.