Multi-Passage Contradiction Flagging Prompt

This prompt is designed for RAG application developers and AI quality engineers who need to surface disagreements in retrieved passages rather than silently resolving or ignoring them. The core job-to-be-done is: given a user query and a set of retrieved passages, produce a structured output that identifies areas of agreement, specific points of contradiction, and the evidence on each side of the disagreement. This is essential for decision-support systems, research assistants, and any application where presenting a false consensus is more harmful than acknowledging uncertainty.

The ideal user is an engineering lead or AI builder integrating this prompt into a RAG pipeline where multiple sources may conflict on facts, interpretations, or recommendations. Required context includes the original user query, the full text of each retrieved passage with source identifiers, and a defined output schema for contradiction flags. This prompt should not be used when the retrieval set is homogeneous, when all sources are known to agree, or when the application requires a single unified answer without exposing internal disagreement. It is also inappropriate for low-latency chat applications where structured contradiction analysis adds unacceptable response time.

Before deploying this prompt, verify that your retrieval pipeline is returning diverse enough sources for contradictions to surface. If your retrieval always returns passages from a single authoritative document or consistently agrees, this prompt will produce empty contradiction arrays and waste tokens. Pair this prompt with a pre-check that assesses whether multiple distinct sources or viewpoints exist in the retrieval set. For high-stakes domains such as medical, legal, or financial applications, always route flagged contradictions to human review before presenting them to end users. The prompt identifies disagreement; it does not resolve which side is correct.

This section provides a copy-ready prompt template for the Multi-Passage Contradiction Flagging task. The template is designed to be dropped into a RAG pipeline after retrieval but before answer generation. It instructs the model to surface disagreements rather than silently harmonizing them, producing structured output that downstream systems or human reviewers can act on. The template uses square-bracket placeholders that you replace with your specific passages, output schema requirements, and operational constraints before sending the request to the model.

textCopy
You are a contradiction detection system. Your job is to compare multiple text passages and identify where they agree, where they disagree, and where they address different aspects of the same topic without direct conflict.

## INPUT PASSAGES
[PASSAGES]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "areas_of_agreement": [
    {
      "claim": "A specific factual claim that appears consistently across sources",
      "supporting_passage_ids": ["passage_1", "passage_3"],
      "confidence": "high | medium | low"
    }
  ],
  "contradictions": [
    {
      "contradiction_id": "contra_1",
      "topic": "Brief label for the disputed point",
      "position_a": {
        "claim": "What one side asserts",
        "passage_ids": ["passage_2"],
        "evidence_excerpt": "Relevant quote from the passage"
      },
      "position_b": {
        "claim": "What the other side asserts",
        "passage_ids": ["passage_4"],
        "evidence_excerpt": "Relevant quote from the passage"
      },
      "contradiction_type": "direct_factual | scope_difference | temporal | definitional | methodological",
      "resolvability": "resolvable_with_more_context | potentially_reconcilable | irreconcilable_given_evidence",
      "severity": "critical | moderate | minor"
    }
  ],
  "independent_points": [
    {
      "point": "A claim made by only one source that others do not address",
      "passage_ids": ["passage_5"],
      "relationship_to_other_passages": "complementary | orthogonal | unverifiable"
    }
  ],
  "synthesis_notes": "A 2-3 sentence summary of the overall evidence picture, explicitly noting whether the passages converge, diverge, or address different questions.",
  "requires_human_review": true,
  "human_review_reason": "Explain why human review is recommended, if applicable."
}

## CONSTRAINTS
[CONSTRAINTS]

## INSTRUCTIONS
1. Compare every passage against every other passage for factual claims.
2. Only flag a contradiction when passages make claims that cannot all be true simultaneously.
3. Do not flag differences in emphasis, wording, or level of detail as contradictions.
4. When passages address different subtopics without overlapping claims, classify them as independent points, not contradictions.
5. For each contradiction, extract the exact evidence text that supports each position.
6. If no contradictions exist, return an empty contradictions array and explain why the passages are consistent.
7. Set requires_human_review to true if any contradiction has severity critical or if any claim affects a [RISK_DOMAIN] decision.
8. Do not invent claims not present in the passages.
9. If passages are too vague to compare, note this in synthesis_notes and return empty arrays.

Adapting the template: Replace [PASSAGES] with your retrieved text, formatted with clear passage identifiers (e.g., passage_1: "..."). Replace [CONSTRAINTS] with domain-specific rules such as 'Do not flag contradictions in opinion statements' or 'Treat passages from peer-reviewed sources as higher authority.' Replace [RISK_DOMAIN] with your application's risk context (e.g., 'clinical', 'financial', 'legal') to trigger human review for critical contradictions. If your application requires a different output format, modify the JSON schema but preserve the core fields: areas of agreement, contradictions with evidence excerpts, and a resolvability assessment.

What to do next: After copying this template, pair it with the eval criteria in the Testing and Evaluation section of this playbook. Run the prompt against a golden dataset of passage sets with known contradictions and known-consistent sets to calibrate your contradiction detection threshold. If the model produces false flags (flagging differences in wording as contradictions), add a constraint or few-shot example. If it misses contradictions, lower the detection threshold by adding an instruction like 'Flag even subtle factual disagreements.' For high-stakes domains, always route outputs with requires_human_review: true to a review queue before the contradiction report reaches end users.

This prompt is designed as a post-retrieval, pre-response analysis step in a RAG pipeline. It should be invoked after the retrieval system returns a set of candidate passages but before the final answer is generated for the user. The prompt expects a structured input containing the user's query and the full text of each retrieved passage, each with a unique identifier. The model's job is to produce a structured contradiction report, not to answer the user's question. This separation of concerns—analysis first, synthesis later—is critical for production reliability. Do not combine contradiction detection with answer generation in a single prompt; the tasks compete for attention and degrade both outputs.

Wire this prompt into your application as a dedicated analysis step with its own validation layer. The output schema should be strictly enforced: expect a JSON object with areas_of_agreement (array of strings), contradictions (array of objects, each containing point_of_contention, side_a with source_ids and claim, and side_b with source_ids and claim), and unresolvable_flags (array of objects with description and source_ids). Implement a post-processing validator that checks: (1) every source_id referenced in the output exists in the input passages, (2) no contradiction entry is self-contradictory (same source on both sides), (3) areas_of_agreement entries do not appear in contradictions, and (4) the output is valid JSON. If validation fails, retry once with the error message appended to the prompt as a correction instruction. If the retry also fails, log the failure and escalate to a human review queue rather than silently proceeding.

Model choice matters here. Use a model with strong reasoning and structured output capabilities, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may conflate contradiction with simple difference or fail to maintain the output schema under pressure. Set temperature to 0 or near-zero to maximize consistency across runs. If your retrieval system returns more than 8-10 passages, pre-filter to the most relevant and authoritative sources before invoking this prompt; the model's contradiction detection accuracy degrades as passage count increases. For high-stakes domains such as legal, medical, or financial applications, always route outputs with unresolvable_flags to human review before any downstream answer generation. Log every contradiction report alongside the input passages and final user-facing answer for auditability.

The output of this prompt should feed into your answer generation step. If contradictions are detected, your synthesis prompt should receive the contradiction report as additional context and be instructed to surface disagreements rather than fabricate consensus. If unresolvable_flags are present, the synthesis prompt should refuse to produce a unified answer and instead present the conflicting evidence to the user with an explanation of why resolution is not possible from the available sources. Test this end-to-end flow with a golden dataset of known contradictory passage sets, measuring both contradiction recall (did the prompt catch all known contradictions?) and false-flag rate (did it invent contradictions where none exist?). A healthy target is recall above 0.85 and false-flag rate below 0.10 before you ship to users.