Conflicting Passage Pair Identification Prompt

This prompt identifies pairs of passages within a bounded retrieval set that directly contradict each other on a specific claim. It is built for retrieval evaluation engineers and QA pipeline builders who need to measure whether their retrieval systems surface conflicting evidence or hide disagreement. The core job-to-be-done is not open-domain contradiction search; it is a targeted diagnostic: given a claim and a set of candidate passages, which passages cannot both be true? The prompt handles both explicit textual conflicts and implicit contradictions that require inference, but it instructs the model to label the conflict type so your harness can route implicit cases for human review.

Use this prompt when you have a pre-retrieved set of passages and a single claim to test against them. The ideal user is running a RAG evaluation harness, a retrieval quality audit, or a pre-production QA gate. Required context includes the claim string, the passage set with identifiers, and an output schema that captures the conflicting pair, the conflict type, and a brief explanation. Do not use this prompt for open-ended document exploration, for comparing more than two passages at a time, or for determining which side of a conflict is correct. It is a detection tool, not a resolution or ranking tool. For resolution, pair this with a conflict resolution decision prompt; for ranking, use an evidence weighting prompt after conflicts are surfaced.

Before wiring this into a production pipeline, define your tolerance for implicit contradictions. Explicit textual conflicts—where two passages state opposite facts—are the high-confidence case. Implicit conflicts, where inference is required, carry higher error risk. Your harness should log the conflict type from every response, route implicit cases to a review queue, and track false-positive rates over time. If your domain involves nuanced methodological disagreements rather than factual contradictions, consider the source disagreement classification prompt instead. Start with a golden dataset of known conflicting and non-conflicting passage pairs to calibrate the model's sensitivity before relying on this prompt in an automated gate.

This prompt template is designed to be copied directly into your AI harness. It instructs the model to analyze a set of passages against a given claim and identify pairs that directly contradict each other. The model is required to return structured JSON with conflict pairs, conflict types, and explanations grounded in the passage text. Before using this prompt, ensure you have a clear claim statement and a set of retrieved passages that may contain conflicting information. This prompt is not suitable for identifying subtle differences in scope or methodology—it targets direct factual contradictions.

textCopy
You are an evidence conflict detection system. Your task is to identify pairs of passages that directly contradict each other regarding a specific claim.

## INPUT
Claim: [CLAIM]

Passages:
[PASSAGES]

## OUTPUT_SCHEMA
Return a JSON object with the following structure:
{
  "conflict_pairs": [
    {
      "passage_a_id": "string",
      "passage_b_id": "string",
      "conflict_type": "direct_contradiction | mutually_exclusive | factual_disagreement",
      "explanation": "string explaining the contradiction, quoting relevant text from both passages",
      "claim_relationship": "supports | refutes | partially_supports | partially_refutes"
    }
  ],
  "no_conflict_pairs": boolean
}

## CONSTRAINTS
- Only identify pairs where passages directly contradict each other on the claim.
- Do not flag differences in scope, methodology, or level of detail as contradictions.
- Each explanation must include verbatim quotes from both passages.
- If no conflicting pairs are found, set no_conflict_pairs to true and return an empty conflict_pairs array.
- Do not invent conflicts that are not explicitly supported by the text.

## EXAMPLES
[EXAMPLES]

To adapt this prompt for your specific use case, replace the [CLAIM] placeholder with the statement you want to verify. The [PASSAGES] placeholder should contain a list of passages, each with a unique identifier and the passage text. For best results, format passages as a JSON array or numbered list with clear IDs. The [EXAMPLES] placeholder should be replaced with 2-3 few-shot examples demonstrating correct conflict identification, including edge cases where passages appear to conflict but actually discuss different aspects of the claim. If your application requires additional output fields—such as confidence scores or source metadata—extend the OUTPUT_SCHEMA section while maintaining the core conflict pair structure. For high-stakes applications, always route outputs through a human review step before taking action on detected conflicts.

This prompt is designed to operate as a batch evaluation step inside a retrieval quality or fact-verification pipeline, not as a real-time user-facing endpoint. The typical integration pattern is: retrieve a set of passages for a claim, run this prompt over all passage pairs (or a sampled subset for large retrieval sets), collect contradiction verdicts, and feed results into downstream metrics such as retrieval precision, conflict density, or hallucination risk scores. Because the prompt performs pairwise comparison, the harness must manage the combinatorial explosion—for N passages, there are N*(N-1)/2 pairs—so implement a pair budget (e.g., top-K by retrieval score) or use a lightweight pre-filter to skip obviously non-conflicting pairs before invoking the LLM.

Validation and output contracts are critical here. The prompt should return structured JSON with fields like passage_a_id, passage_b_id, contradiction_detected (boolean), contradiction_type (enum: explicit, implicit, none), conflicting_claim, and explanation. Implement a post-processing validator that checks: (1) both passage IDs exist in the input set, (2) contradiction_detected is consistent with contradiction_type, (3) the conflicting_claim is a non-empty string when contradiction is detected, and (4) the explanation references specific text spans from both passages. Failed validations should trigger a single retry with the validation error message appended to the prompt; if the retry also fails, log the pair for human review rather than silently accepting a malformed output. For high-stakes domains (legal, clinical, financial), route all detected contradictions to a human review queue before they influence downstream decisions.

Model choice and latency planning depend on your throughput requirements. For offline evaluation pipelines processing thousands of passage pairs, batch the pairs into requests of 10–20 comparisons per call to amortize prompt overhead, and use a fast model (e.g., GPT-4o-mini, Claude Haiku) for the initial sweep. Reserve a stronger model (GPT-4o, Claude Sonnet) for edge-case re-evaluation when the primary model returns low-confidence or ambiguous results. Log every comparison with the model version, prompt template hash, input passage IDs, raw output, validation status, and final verdict. This trace data becomes your golden evaluation set for regression testing when you iterate on the prompt or switch models. The most common production failure mode is false positives on paraphrased agreement—two passages saying the same thing in different words flagged as contradictory. Mitigate this by including few-shot examples of paraphrased agreement labeled as contradiction_detected: false and by running periodic spot-checks on a stratified sample of verdicts.

To integrate this into a CI/CD pipeline for prompt regression testing, maintain a curated test suite of 50–100 passage pairs with known contradiction labels (including edge cases: implicit contradictions, numerical disagreements, scope mismatches, and paraphrased agreement). Run the prompt against this suite on every prompt change, and fail the pipeline if precision or recall drops below your threshold (suggested starting point: precision ≥ 0.85, recall ≥ 0.80 on explicit contradictions). Pair this with an LLM-as-judge evaluation step that independently reviews a sample of production verdicts for consistency and correctness, feeding discrepancies back into your test suite. Avoid wiring contradiction detection directly into user-facing answer generation without a human-in-the-loop step for high-severity conflicts.