Context Contradiction Detection Prompt for Review

This prompt is designed for a specific and critical job-to-be-done: acting as a safety gate in a Retrieval-Augmented Generation (RAG) pipeline. Its ideal user is a knowledge system engineer or AI product developer who has already retrieved multiple context chunks from a vector database, search index, or document store. The core task is to programmatically decide if the assembled context is internally consistent enough for a downstream LLM to synthesize into a single, coherent answer. Without this check, a model may blend conflicting facts, leading to a confident but incorrect response that erodes user trust and creates a liability. The prompt assumes you have source identifiers (e.g., URLs, document titles, chunk IDs) attached to each piece of retrieved context, which are essential for the attribution step.

You should use this prompt when the cost of a factual error is high and the retrieval step frequently surfaces documents from different versions, authors, or time periods. For example, a technical support copilot might retrieve a deprecated API specification alongside the latest version, or a legal research tool might find two court rulings with opposing conclusions. The prompt's structured output—a contradiction report—is designed to be machine-readable, allowing your application to automatically halt the answer generation step and route the conflict to a human review queue. Do not use this prompt for single-source verification, where the task is to check if a claim is supported by one document. It is also unsuitable for detecting stylistic disagreements or subjective opinion differences where no factual truth is at stake; using it in these scenarios will generate a high rate of false positives, flagging benign differences as critical conflicts and overwhelming your review team.

Before integrating this prompt, ensure your upstream retrieval pipeline reliably returns context chunks with stable, unique source identifiers. A common failure mode is feeding the prompt context that has already been summarized or rephrased, which can obscure the precise factual claims and lead to missed contradictions. The next step after reading this section is to examine the prompt template and its required input variables, particularly how to structure the [CONTEXT] block with clear source attribution. Avoid the temptation to use this as a general-purpose 'fact-checker' on a single document; its strength lies in comparative analysis across multiple sources, and misapplying it will lead to unreliable and noisy outputs.

This prompt is designed to be dropped directly into a RAG pipeline after retrieval and before answer generation. It compares multiple source passages, identifies factual conflicts, and produces a structured contradiction report that can be routed to a human review queue. The template uses square-bracket placeholders that you replace with your actual retrieved context, output schema requirements, and escalation thresholds before deployment.

textCopy
You are a contradiction detection system operating within a RAG pipeline. Your job is to compare multiple retrieved source passages and identify factual conflicts before answer generation proceeds.

## INPUT

[RETRIEVED_CONTEXT]

## TASK

1. Extract all factual claims from each source passage. A factual claim is a verifiable statement about entities, events, dates, quantities, relationships, or properties.

2. Compare claims across sources and identify contradictions. A contradiction exists when two or more sources assert mutually exclusive facts about the same entity, event, or property.

3. For each contradiction found, produce a structured entry containing:
   - The conflicting claims with exact source text excerpts
   - Source attribution for each claim (source ID, passage index)
   - The type of contradiction: [CONTRADICTION_TYPES]
   - A severity rating: [SEVERITY_SCALE]
   - Whether the contradiction can be resolved by preferring a more authoritative or recent source

4. Classify the overall context set as:
   - CONSISTENT: No factual conflicts detected
   - MINOR_CONFLICT: Conflicts exist but are resolvable or low-impact
   - MAJOR_CONFLICT: Unresolvable conflicts that would affect answer quality
   - UNVERIFIABLE: Insufficient evidence to determine consistency

## CONSTRAINTS

- Only flag factual contradictions, not differences in opinion, emphasis, or level of detail
- Do not fabricate contradictions where sources can be reasonably reconciled
- If sources use different terminology for the same entity, treat them as referring to the same entity unless context clearly indicates otherwise
- Mark temporal contradictions separately from static factual contradictions
- If [RISK_LEVEL] is HIGH, escalate any contradiction regardless of severity

## OUTPUT FORMAT

Return a JSON object matching this schema:

[OUTPUT_SCHEMA]

## EXAMPLES

[EXAMPLES]

Adaptation guidance: Replace [RETRIEVED_CONTEXT] with your actual retrieved passages, including source IDs and passage indices. Define [CONTRADICTION_TYPES] as a closed set matching your domain (e.g., ["temporal", "quantitative", "entity_identity", "causal", "categorical"]). Set [SEVERITY_SCALE] to match your escalation thresholds (e.g., ["low", "medium", "high", "critical"]). Provide [OUTPUT_SCHEMA] as the exact JSON schema your downstream systems expect. Include [EXAMPLES] with at least two few-shot demonstrations: one showing a clear contradiction with proper attribution, and one showing sources that appear contradictory but are actually reconcilable. Set [RISK_LEVEL] based on your application context—use HIGH for regulated domains where any contradiction must trigger human review.

Before deploying, validate that the prompt produces parseable JSON matching your schema on at least 20 test cases with known contradictions. Measure contradiction recall (did it catch all known conflicts?) and false conflict rate (did it flag non-contradictions?). If false conflict rate exceeds 10%, add more few-shot examples showing reconcilable differences. Wire the output into a human review queue that presents the contradiction summary, source excerpts, and severity classification to a reviewer who can approve, override, or request re-retrieval before answer generation proceeds.

The Context Contradiction Detection Prompt is designed to sit between retrieval and answer generation in a RAG pipeline. Its job is to inspect the set of retrieved documents for factual conflicts before the model synthesizes a response. In a production harness, this prompt should be called as a dedicated validation step after retrieval returns its top-k results but before those results are passed to the final generation prompt. This separation allows the contradiction detection to run with a different model, temperature, or timeout than the main generation step, and it creates a clean decision boundary: if contradictions are found, the pipeline can either halt for human review or proceed with a contradiction-aware disclaimer appended to the final answer.

The implementation should wrap the prompt call in a structured validation layer. The prompt returns a JSON object with fields like contradiction_detected, contradiction_summary, conflicting_sources, and severity. Your harness must validate this schema on every response. If the model returns malformed JSON, retry once with a stricter instruction appended to the prompt. If the retry also fails, log the raw output and escalate the entire retrieval set to a human review queue rather than proceeding with potentially flawed context. For high-stakes domains such as clinical or legal applications, consider routing any detection with severity: "high" directly to a human reviewer and blocking automated answer generation entirely. The harness should also log every contradiction detection event with the retrieval query, the document IDs, the contradiction summary, and the final routing decision for audit and eval purposes.

Model choice matters here. Contradiction detection is a reasoning task that benefits from strong instruction-following models. In testing, smaller or quantized models often produce false positives by conflating complementary information with contradiction. Start with a capable model like Claude 3.5 Sonnet or GPT-4o, then experiment with smaller models only if latency or cost demands it, and always run a calibration eval against human-annotated contradiction labels before switching. Set temperature to 0 or near-zero to maximize consistency. The prompt should be called with a timeout appropriate to your retrieval volume; if contradiction detection adds more than 500ms to your pipeline, consider batching multiple retrieval sets or using a faster model for low-priority queries. For RAG systems with very large context windows, truncate the retrieved passages to the most relevant sections before passing them to this prompt to avoid token waste and reduce false positives from out-of-context snippets.

The human review integration is the most critical part of the harness. When the prompt flags a contradiction, your system should create a review item containing the original user query, the full text of the conflicting sources, the contradiction summary generated by the model, and a direct link to the source documents if available. Do not rely on the model's contradiction summary alone; the human reviewer needs the raw evidence to make an independent judgment. Route review items to a queue that supports prioritization by severity and tracks time-to-resolution. Over time, collect reviewer feedback on whether each flagged contradiction was a true positive or false positive. This feedback loop is essential for tuning the prompt's instructions, adjusting severity thresholds, and measuring the eval metrics described in the topic overview: contradiction recall and false conflict rate. Without this closed loop, the prompt will drift in effectiveness as your document corpus and query patterns evolve.