Contradiction Detection Harness Prompt for RAG Pipelines

This prompt is designed for RAG system builders who need to add a post-retrieval contradiction check before generating an answer. When a user query triggers retrieval of multiple passages, those passages may contain conflicting facts, figures, or interpretations. This prompt acts as a harness: it takes a set of retrieved passages with their citation metadata and produces a structured contradiction signal. The signal can then be used to suppress an answer, qualify it with a warning, or route the query for human review. Use this when you need a machine-readable contradiction verdict integrated directly into your RAG pipeline, not a human-readable explanation. It assumes you already have a retrieval step producing passages with source identifiers.

Do not use this prompt for open-ended document comparison, single-document proofreading, or claim extraction without retrieval context. It is not a general fact-checking tool—it expects a specific input shape: a user query paired with multiple retrieved passages, each carrying a unique source ID. The output is a structured JSON verdict, not prose. If your goal is to explain contradictions to an end user, pair this prompt with a downstream summarization step. If your latency budget is under 200ms, consider batching smaller passage sets or using a lighter classification model for the contradiction check before invoking this more thorough analysis.

Before deploying, validate that your retrieval step returns passages with stable, unique identifiers. The prompt's utility depends entirely on traceable provenance—without source IDs, you cannot act on the contradiction signal. Start by wiring this prompt after retrieval and before answer generation, then add an eval suite that measures both contradiction detection recall and false-positive rates on known-consistent passage sets. Avoid using this prompt on single-source retrievals; the contradiction check is meaningless without at least two distinct sources to compare.

This prompt template is the core instruction set for a post-retrieval contradiction detection harness. It is designed to be invoked after your retrieval system has returned a set of candidate passages for a user query. The prompt forces the model to perform pairwise comparison of the provided passages, identify direct contradictions, and output a structured signal that your application can use to suppress, qualify, or flag the final generated answer. The primary job is not to answer the user query, but to audit the evidence for internal consistency. Use this when the cost of presenting conflicting information to a user is high, such as in financial analysis, medical summarization, or legal research.

textCopy
You are an evidence auditor. Your task is to analyze a set of retrieved text passages for direct factual contradictions. Do not answer the user's original question. Focus only on whether the provided passages conflict with each other on specific, verifiable facts.

## INPUT
User Query: [USER_QUERY]
Retrieved Passages:
[RETRIEVED_PASSAGES]

## TASK
For every pair of passages, determine if they contain a direct contradiction. A direct contradiction exists only if two passages assert mutually exclusive facts about the same specific entity, event, number, date, or claim. Do not flag differences in phrasing, scope, or opinion as contradictions.

## CONSTRAINTS
- Ignore contradictions that are irrelevant to the user query.
- If a contradiction is found, you must cite the exact text from both passages that creates the conflict.
- If no contradictions are found, explicitly state "NO_CONTRADICTIONS_DETECTED".
- Do not hallucinate conflicts. If you are unsure, classify as "NO_CONTRADICTIONS_DETECTED".

## OUTPUT_SCHEMA
Respond with a valid JSON object conforming to this schema:
{
  "contradictions": [
    {
      "passage_ids": ["id_1", "id_2"],
      "conflict_type": "direct_contradiction",
      "conflicting_claim_1": "Exact excerpt from passage 1",
      "conflicting_claim_2": "Exact excerpt from passage 2",
      "explanation": "Brief explanation of why these are mutually exclusive."
    }
  ],
  "status": "CONTRADICTIONS_FOUND" | "NO_CONTRADICTIONS_DETECTED"
}

To adapt this template, you must format your retrieved passages into the [RETRIEVED_PASSAGES] placeholder as a string that clearly separates each passage with a unique identifier. A robust pattern is to prepend each passage with [ID: chunk_01] before injecting it. The [USER_QUERY] placeholder should contain the original user question to provide relevance context, preventing the model from flagging trivial or off-topic disagreements. For high-stakes domains, add a [RISK_LEVEL] placeholder and adjust the instructions to require higher confidence thresholds or to route CONTRADICTIONS_FOUND outputs directly to a human review queue. The output schema is intentionally flat to make parsing reliable; your application harness should validate that the JSON is well-formed and that the status field is present before proceeding to answer generation.

Integrating the contradiction detection prompt into a production RAG pipeline requires treating it as a post-retrieval verification step, not a standalone classifier. The prompt should receive a batch of retrieved passage pairs—each pair consisting of a claim-bearing passage and a potentially conflicting passage—along with their citation metadata. The application layer is responsible for constructing these pairs before calling the model, typically by grouping retrieved chunks by the claim they are meant to support or refute. The model returns structured contradiction verdicts, which the harness then validates against an expected output schema before the results influence downstream answer generation, suppression, or qualification logic.

The implementation harness must enforce a strict output contract. Define a JSON schema that includes fields for claim_id, conflicting_passage_id, contradiction_type (e.g., direct_contradiction, partial_mismatch, temporal_inconsistency, numerical_disagreement, compatible), confidence_score (0.0–1.0), explanation, and evidence_excerpts from both passages. After each model call, validate the response against this schema. If validation fails, implement a retry loop with a maximum of two additional attempts, appending the validation error message to the prompt context. Log every validation failure and retry for observability. For high-stakes domains such as healthcare or legal review, route any contradiction with a confidence score between 0.4 and 0.7 to a human review queue, and suppress or flag the affected answer segment rather than presenting it as resolved.

Latency and cost are critical when adding contradiction checks to a RAG pipeline. Batch multiple passage pairs into a single prompt call where possible, but cap the batch size based on your latency budget—start with 5–10 pairs per call and measure end-to-end latency. Use a faster model (e.g., GPT-4o-mini, Claude Haiku) for initial contradiction screening, and escalate ambiguous pairs to a more capable model only when the confidence score falls in the uncertain range. Cache contradiction verdicts for identical passage pairs to avoid redundant computation. Finally, instrument the harness with metrics: contradiction detection rate, validation failure rate, retry count, human escalation rate, and p95 latency per batch. These metrics will tell you whether the contradiction check is adding value or just adding noise to your pipeline.