Inferensys

Prompt

Cross-Document Contradiction Detection Prompt Template

A practical prompt playbook for using Cross-Document Contradiction Detection Prompt Template in production AI workflows.
Operations team reviewing AI workflow automation on laptop, workflow builder visible, casual office setup.
PROMPT PLAYBOOK

When to Use This Prompt

A practical guide to deploying the cross-document contradiction detection prompt in a production verification pipeline, including its ideal job, required inputs, and critical limitations.

This prompt is designed for verification pipeline developers who need to compare factual claims across two or more source documents and identify genuine contradictions. Its primary job-to-be-done is pairwise comparison logic: given a set of pre-extracted claims from Document A and Document B, the prompt produces a structured report classifying each claim pair as a direct contradiction, a partial mismatch, or a compatible statement. The ideal user is an engineer integrating this prompt into a fact-checking pipeline's contradiction detection stage, sitting after claim extraction and before severity scoring or human review routing. You should use this prompt when you have already decomposed source documents into atomic, verifiable claims and need systematic conflict surfacing, not when you are starting from raw, unstructured text.

Do not use this prompt for single-document self-consistency checks without adapting the instructions; it is optimized for cross-document comparison and may miss rhetorical devices or conditional logic that a dedicated internal-consistency prompt would catch. It is also unsuitable for temporal sequence validation, where event ordering and date normalization require specialized reasoning, or for numerical tolerance analysis, where value comparisons need explicit unit conversion and significant-figure handling. If your source documents contain tables, charts, or images, extract and normalize those claims into text before invoking this prompt, as it lacks native multimodal reasoning. For high-stakes domains like healthcare or legal review, always route the prompt's output to a human reviewer and ensure the upstream claim extraction step preserves verbatim source text to ground the contradiction verdicts in evidence.

Before wiring this prompt into your application, confirm that your claim extraction step produces clean, atomic claims with stable identifiers and source provenance. The prompt expects claims as structured input, not raw paragraphs. If your pipeline skips this step, you will see high false-positive rates from the model comparing paraphrases rather than factual content. Next, review the output schema and evaluation criteria in the full playbook to integrate validation checks, retry logic, and human review routing. Start with a small, labeled dataset of known contradictions and non-contradictions to calibrate the prompt's conflict type classification thresholds before scaling to production volumes.

PRACTICAL GUARDRAILS

Use Case Fit

Where this prompt works and where it does not. Cross-document contradiction detection requires pairwise comparison logic, clear evidence boundaries, and tolerance for paraphrasing. Use these cards to decide if this template fits your verification pipeline stage.

01

Good Fit: Multi-Source Fact-Checking Pipelines

Use when: you have two or more source documents making factual claims about the same topic and need structured contradiction reports with claim pairs, conflict types, and evidence excerpts. Guardrail: ensure each source is independently ingested and has stable identifiers before running pairwise comparison.

02

Bad Fit: Single-Document Internal Consistency

Avoid when: your task is finding self-contradictions within one long document. This template is optimized for cross-document comparison. Guardrail: use the Logical Conflict Extraction Prompt for single-document inconsistency detection instead.

03

Required Inputs: Claim-Anchored Source Pairs

What to watch: the prompt needs pre-extracted claims or clearly scoped document segments, not raw document dumps. Feeding full documents without claim boundaries produces noisy, uncalibrated outputs. Guardrail: run claim extraction before contradiction detection. Pass claim text, source ID, and a short context window per claim.

04

Operational Risk: Paraphrase vs. Genuine Contradiction

What to watch: the model may flag paraphrased statements as contradictions when wording differs but meaning aligns. This is the most common false-positive source. Guardrail: include explicit instructions to distinguish semantic equivalence from factual disagreement. Add eval checks with paraphrase test cases.

05

Operational Risk: Missing Evidence Blind Spots

What to watch: the prompt can only detect contradictions between provided sources. It cannot flag claims that lack evidence entirely. Guardrail: pair this prompt with an Unsupported Statement Detection step upstream. Never treat absence of contradiction as confirmation of truth.

06

Scale Constraint: Pairwise Explosion

What to watch: comparing every claim against every other claim across N documents creates O(N²) comparisons. At scale, latency and cost become prohibitive. Guardrail: pre-filter claims by topic, entity, or time window before pairwise comparison. Use clustering or embedding similarity to reduce the comparison set.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable, implementation-ready prompt for detecting contradictions between claims in two or more source documents.

This template is designed to be copied directly into your verification pipeline. It instructs the model to act as a contradiction detection engine, comparing a set of target claims against a corpus of source documents. The prompt forces structured, machine-readable output that includes the conflicting claim pair, a precise conflict type classification, verbatim evidence excerpts, and a confidence score. Before using this template, ensure your upstream pipeline has already extracted discrete, atomic claims from your source documents using a dedicated claim extraction prompt. Feeding raw, unprocessed documents will produce noisy and unreliable results.

text
You are a contradiction detection engine. Your task is to compare a set of [TARGET_CLAIMS] against a corpus of [SOURCE_DOCUMENTS] to identify direct contradictions. A contradiction exists only when two statements cannot both be true at the same time. Do not flag differences in phrasing, scope, or level of detail as contradictions.

# INPUT

## TARGET CLAIMS
A JSON array of claim objects, each with an `id` and `text` field.
```json
[TARGET_CLAIMS]

SOURCE DOCUMENTS

A JSON array of document objects, each with a doc_id and text field.

json
[SOURCE_DOCUMENTS]

TASK

For each claim in [TARGET_CLAIMS], search the [SOURCE_DOCUMENTS] for any statement that directly contradicts it. A contradiction must meet these criteria:

  1. Logical Incompatibility: The two statements assert mutually exclusive facts.
  2. Subject Alignment: The statements must be about the same entity, event, or metric.
  3. Temporal Alignment: The statements must refer to the same time period, or one must assert a state that is impossible given the other's timeframe.

CONSTRAINTS

  • Ignore Paraphrasing: Do not flag a claim as contradictory if the source document expresses the same fact using different words.
  • Ignore Scope Differences: A general statement in one document does not contradict a more specific statement in another unless the specific statement is a logical subset that is directly negated.
  • Ignore Hedging: A statement containing "may," "could," or "reportedly" does not directly contradict a definitive statement, but flag it as a POTENTIAL_MISMATCH.
  • Evidence is Mandatory: You must provide a verbatim excerpt from the source document for every flagged contradiction.

OUTPUT SCHEMA

You must return a valid JSON object with the following structure. Do not include any text outside the JSON object.

json
{
  "contradictions": [
    {
      "claim_id": "string",
      "claim_text": "string",
      "conflicting_doc_id": "string",
      "conflicting_excerpt": "string",
      "conflict_type": "DIRECT_CONTRADICTION | PARTIAL_MISMATCH | POTENTIAL_MISMATCH",
      "confidence": 0.0,
      "rationale": "A brief explanation of why these statements cannot both be true."
    }
  ],
  "unmatched_claim_ids": ["list of claim IDs with no contradiction found"]
}

RISK LEVEL

[RISK_LEVEL]

FINAL INSTRUCTION

Execute the comparison now. Return only the JSON object specified in the OUTPUT SCHEMA.

To adapt this template, replace the square-bracket placeholders with your data. [TARGET_CLAIMS] should be a JSON array of the claims you want to verify, and [SOURCE_DOCUMENTS] should be a JSON array of the evidence documents. The [RISK_LEVEL] field is a control mechanism: set it to HIGH to instruct the model to default to flagging uncertain cases for human review, or STANDARD for automated pipelines. After receiving the output, you must validate the JSON schema and verify that every conflicting_excerpt is a verbatim substring of the provided source documents to prevent hallucinated evidence. For high-stakes domains like healthcare or finance, route all outputs with a confidence score below 0.9 for mandatory human review before any action is taken.

IMPLEMENTATION TABLE

Prompt Variables

Replace each placeholder before sending the prompt to the model. Validation notes describe checks to apply in your application harness before calling the LLM.

PlaceholderPurposeExampleValidation Notes

[DOCUMENT_A]

Full text of the primary source document to compare

The defendant was at the scene at 10:15 PM according to the police report.

Required. Non-empty string. Max 8000 tokens. Must be distinct from [DOCUMENT_B] content.

[DOCUMENT_B]

Full text of the secondary source document to compare against

The witness stated the defendant arrived at 11:00 PM.

Required. Non-empty string. Max 8000 tokens. Reject if identical to [DOCUMENT_A].

[DOCUMENT_METADATA_A]

Source identifier, date, and authority context for Document A

{"source": "Police Report #2024-081", "date": "2024-03-15", "authority": "law enforcement"}

Required. Valid JSON object with source, date, and authority fields. Date must parse as ISO 8601.

[DOCUMENT_METADATA_B]

Source identifier, date, and authority context for Document B

{"source": "Witness Statement #WS-042", "date": "2024-03-16", "authority": "eyewitness"}

Required. Valid JSON object with source, date, and authority fields. Date must parse as ISO 8601.

[CONTRADICTION_TYPES]

List of contradiction categories the model should classify into

["direct_contradiction", "partial_mismatch", "temporal_inconsistency", "numerical_disagreement", "logical_conflict"]

Required. Non-empty array of strings. Each value must be from the allowed enum: direct_contradiction, partial_mismatch, temporal_inconsistency, numerical_disagreement, logical_conflict.

[OUTPUT_SCHEMA]

Expected JSON structure for the contradiction report

{"contradictions": [{"claim_a": "...", "claim_b": "...", "type": "direct_contradiction", "confidence": 0.92, "explanation": "..."}]}

Required. Valid JSON Schema or example object. Must include contradictions array, claim_a, claim_b, type, confidence, and explanation fields.

[CONFIDENCE_THRESHOLD]

Minimum confidence score for reporting a contradiction

0.75

Required. Float between 0.0 and 1.0. Contradictions below this threshold should be suppressed or flagged for human review.

[MAX_CONTRADICTIONS]

Upper limit on number of contradiction pairs to return

10

Optional. Integer between 1 and 50. Defaults to 10 if not provided. Prevents runaway output on long documents.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the cross-document contradiction detection prompt into a production verification pipeline with validation, retries, and human review.

This prompt is designed to be a stateless function inside a larger verification pipeline. It expects pre-extracted claims and their source document contexts as input, not raw documents. The calling application is responsible for claim extraction, document chunking, and evidence pairing before invoking this prompt. A typical harness wraps the prompt in a service that accepts a batch of claim pairs, fans out to the LLM, collects structured responses, and routes results based on conflict severity and confidence scores. Because contradiction detection is a pairwise comparison task, the harness should implement batching with concurrency limits to stay within rate limits while processing document pairs efficiently. For high-stakes domains like legal or clinical verification, the harness must enforce a human-review gate for all detected contradictions above a configurable severity threshold before any downstream action is taken.

The implementation must include schema validation on every response. The prompt's output contract specifies a JSON object with fields for claim_a, claim_b, conflict_type, confidence, evidence_excerpts, and explanation. Before accepting a model response, validate that conflict_type is one of the enumerated values (direct_contradiction, partial_mismatch, temporal_inconsistency, numerical_disagreement, compatible, unrelated), that confidence is a float between 0.0 and 1.0, and that evidence_excerpts contains non-empty strings that can be traced back to the provided source documents. Failed validations should trigger a retry loop with the original prompt plus the validation error message appended as feedback. Cap retries at three attempts, then escalate to a human review queue with the full context attached. Log every attempt, the raw model output, validation errors, and final disposition for auditability and prompt debugging. For model selection, use a model with strong instruction-following and structured output capabilities; GPT-4o or Claude 3.5 Sonnet are suitable defaults. Avoid smaller or older models that may collapse conflict_type into binary true/false or omit required fields under token pressure.

When integrating with a RAG pipeline, the harness should pre-filter document pairs before invoking this prompt. Only pass claim pairs where both documents have been retrieved with sufficient relevance scores, and where the claims share a common topic or entity. Blind pairwise comparison across all document pairs in a large corpus is computationally prohibitive and generates excessive false positives from unrelated claims. Implement a lightweight pre-screening step—either a simpler prompt or a vector similarity threshold—to select candidate pairs for detailed contradiction analysis. The harness should also track false positive patterns over time: if certain conflict types or source pairs consistently generate contradictions that human reviewers overturn, feed those examples back into few-shot examples or adjust confidence thresholds. Finally, never treat the model's confidence score as a calibrated probability without empirical validation against human judgments on your specific data distribution. Run periodic eval batches against a golden dataset of known contradictions and non-contradictions to monitor drift and recalibrate thresholds.

IMPLEMENTATION TABLE

Expected Output Contract

Structured output fields for the cross-document contradiction detection prompt. Use this contract to validate model responses before they enter downstream verification pipelines.

Field or ElementType or FormatRequiredValidation Rule

contradiction_id

string (UUID v4)

Must be a valid UUID v4 string. Reject if missing or malformed.

claim_pair

array of two objects

Array must contain exactly two elements. Each element must have non-empty claim_text and source_document_id fields.

claim_pair[].claim_text

string

Must be a non-empty string. Must be a verbatim or near-verbatim excerpt from the source document. Reject if empty or only whitespace.

claim_pair[].source_document_id

string

Must match a document ID provided in [SOURCE_DOCUMENTS]. Reject if ID is not in the input set.

conflict_type

enum: [direct_contradiction, partial_mismatch, temporal_inconsistency, numerical_disagreement, logical_conflict, no_conflict]

Must be exactly one of the enumerated values. Reject if missing, misspelled, or not in the allowed set.

confidence_score

number (0.0 to 1.0)

Must be a float between 0.0 and 1.0 inclusive. Values below [CONFIDENCE_THRESHOLD] should trigger human review routing.

evidence_excerpts

array of strings

Array must contain at least one non-empty string per claim. Each excerpt must be a direct quote from the corresponding source document. Reject if empty array or placeholder text.

explanation

string

Must be a non-empty string between 20 and 500 characters. Must reference specific conflicting elements from both claims. Reject if generic or circular reasoning detected.

PRACTICAL GUARDRAILS

Common Failure Modes

Cross-document contradiction detection fails in predictable ways. These cards cover the most common production failure modes and the specific guardrails that prevent them.

01

Paraphrase Misclassified as Contradiction

What to watch: The model flags semantically equivalent statements as contradictions because they use different words, sentence structures, or terminology. This is the most common false-positive source in production contradiction pipelines. Guardrail: Add a pre-check instruction requiring the model to restate both claims in a canonical form before comparing. Include few-shot examples of paraphrases that must be classified as 'compatible.' Use an eval set with known paraphrase pairs to measure false-positive rate before deployment.

02

Scope Mismatch Produces False Conflicts

What to watch: One source makes a claim about a specific subset, time window, or condition while another makes a broader claim. The model treats the unqualified broader statement as contradicting the narrower one. Guardrail: Require explicit scope extraction before comparison—time range, population, conditions, and qualifiers must be surfaced. Add a contradiction rule: if scopes do not fully overlap, classify as 'partial mismatch' rather than 'direct contradiction.' Include scope-mismatch examples in few-shot prompts.

03

Hedging and Uncertainty Language Ignored

What to watch: Statements containing 'may,' 'likely,' 'estimates suggest,' or 'preliminary findings indicate' are compared as if they were absolute claims. This generates contradictions where none exist because the uncertainty qualifiers are stripped during comparison. Guardrail: Add an explicit hedging extraction step before contradiction analysis. Classify claims with non-overlapping confidence levels as 'compatible with different certainty' rather than contradictory. Include a hedging glossary in the system prompt with examples of qualifiers that should prevent direct contradiction classification.

04

Temporal Context Collapse

What to watch: Claims from different time periods are compared without accounting for when each was true. A statement accurate in January contradicts a statement accurate in June, but both were correct at their respective times. Guardrail: Require timestamp or effective-date extraction for every claim before comparison. Add a temporal compatibility rule: claims with non-overlapping validity periods are not contradictions unless one explicitly claims the other's timeframe. Use a temporal reasoning validation step that checks whether both claims could be true at different times.

05

Source Authority Blind Comparison

What to watch: The model treats all sources as equally authoritative, flagging contradictions between a primary source and an unreliable secondary source without weighting credibility. This wastes reviewer time on low-quality conflicts. Guardrail: Add source authority metadata to each claim before comparison. Implement a pre-filter that suppresses contradiction flags when one source falls below an authority threshold and the higher-authority source is unambiguous. Route low-authority conflicts to a separate 'low-confidence' queue rather than the main contradiction report.

06

Context Window Truncation Drops Critical Qualifiers

What to watch: When comparing claims from long documents, the surrounding context that contains definitions, scope limitations, or caveats is truncated from the prompt. The model compares stripped claims and generates false contradictions. Guardrail: Implement context-window budgeting that reserves space for surrounding paragraphs, not just extracted claim sentences. Use a retrieval step to pull definitional or scoping passages from each source before comparison. Add a validation check: if a claim's source context was truncated, mark the contradiction verdict as 'context-incomplete' and suppress high-confidence output.

IMPLEMENTATION TABLE

Evaluation Rubric

How to test output quality before shipping. Use this rubric with a golden dataset of claim pairs with known contradiction labels.

CriterionPass StandardFailure SignalTest Method

Contradiction Classification Accuracy

F1 score >= 0.85 on golden dataset of 200+ labeled claim pairs

F1 score < 0.80 or systematic misclassification of partial mismatches as direct contradictions

Run prompt against golden dataset; compute precision, recall, F1 per class; inspect confusion matrix for bias toward over-flagging

Paraphrase Resilience

Zero false positives on paraphrase pairs where meaning is preserved but wording differs

Any paraphrase pair flagged as contradiction or partial mismatch

Include 30+ paraphrase pairs in golden set; assert contradiction_label == 'unrelated' or 'compatible' for all

Conflict Type Classification

Conflict type matches expected label in >= 80% of true-positive cases

Direct contradictions mislabeled as temporal or numerical; type confusion rate > 20%

Subset golden data by conflict type; measure per-type accuracy; flag rows where verdict is correct but type is wrong

Evidence Excerpt Completeness

Every contradiction verdict includes at least one excerpt from each source document

Missing excerpt field, null excerpt, or excerpt that does not contain the conflicting claim text

Parse output JSON; assert [EVIDENCE_EXCERPT_A] and [EVIDENCE_EXCERPT_B] are non-null strings with length > 20 chars

Confidence Score Calibration

Mean confidence >= 0.7 for correct verdicts; mean confidence <= 0.5 for incorrect verdicts

High confidence on false positives or low confidence on obvious contradictions

Bucket predictions by confidence decile; compute accuracy per bucket; expect monotonic increase

Output Schema Compliance

100% of outputs parse as valid JSON matching the defined [OUTPUT_SCHEMA]

Missing required fields, wrong types, extra fields, or unparseable JSON

Validate all outputs against JSON Schema; count schema violations; reject any run with > 0 violations

Null Handling for Unverifiable Pairs

Returns verdict 'unrelated' or 'insufficient_context' when claims lack overlapping entities or topics

Forced contradiction label on clearly unrelated claim pairs

Include 20+ unrelated claim pairs in golden set; assert verdict is not 'direct_contradiction' or 'partial_mismatch'

Latency Budget Compliance

Median response time <= 3 seconds per claim pair under normal load

P95 latency > 10 seconds or timeout rate > 2%

Measure end-to-end latency across 100+ test runs; log outliers; flag if retry rate exceeds threshold

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a frontier model (GPT-4o, Claude 3.5 Sonnet) and minimal validation. Focus on getting the contradiction structure right before adding pipeline complexity. Start with two short documents and a simple output schema:

code
[SYSTEM]
You are a contradiction detector. Compare the two provided documents and identify direct factual contradictions.

[DOCUMENT_A]
[INPUT_DOC_A]

[DOCUMENT_B]
[INPUT_DOC_B]

[OUTPUT_SCHEMA]
{
  "contradictions": [
    {
      "claim_a": "...",
      "claim_b": "...",
      "conflict_type": "direct_contradiction | partial_mismatch | numerical_disagreement",
      "explanation": "..."
    }
  ]
}

Watch for

  • Paraphrased statements flagged as contradictions (false positives from synonym use)
  • Missing conflict_type classification when claims are only partially overlapping
  • Output schema drift when the model adds extra fields or changes field names
  • No handling of "no contradictions found" case — add an explicit empty state instruction
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.