Multi-Source Conflict Detection Prompt Template

This prompt is designed for knowledge base operators and RAG pipeline builders who need to detect when retrieved documents make contradictory claims about the same fact, entity, or event. Use it when your retrieval system returns multiple sources and you must surface disagreements rather than silently averaging or ignoring them. The prompt expects pre-retrieved document chunks with source identifiers and outputs a structured conflict report with source pairs, claim excerpts, conflict type classification, and severity indicators.

This is a detection prompt, not a resolution prompt. It identifies contradictions but does not decide which source is correct. Pair it with source credibility weighting or temporal conflict resolution prompts when you need to resolve conflicts after detection. The prompt works best with 3-15 document chunks. Fewer than 3 sources makes conflict detection trivial. More than 15 may exceed context windows or produce noisy results.

Do not use this prompt when you have a single source, when all sources are known to be authoritative and consistent, or when your application should present a unified answer without exposing internal disagreements. For high-stakes domains such as healthcare, legal, or finance, always route detected conflicts to human review before surfacing them to end users. The prompt identifies contradictions but does not assess their materiality to the user's question—you must apply domain-specific severity thresholds downstream.

This template is designed to be dropped directly into your system prompt or user message template within a RAG pipeline. It instructs the model to act as a contradiction detector, comparing claims across a set of provided documents. The prompt is structured to force a systematic, claim-by-claim comparison rather than a superficial summary, which is critical for high-recall conflict detection. Before using this template, ensure your retrieval step has already gathered a candidate set of documents that are topically related; this prompt does not perform retrieval, only analysis of the provided context.

markdownCopy
You are a contradiction detection engine. Your task is to analyze the provided [DOCUMENTS] and identify all pairs of claims that directly contradict each other. A contradiction exists only when two claims cannot both be true at the same time and in the same respect.

## INPUT
[DOCUMENTS]: A JSON array of document objects. Each object has an `id` (string), `text` (string), and optional `metadata` (object with `date`, `source`, `authority`).

## TASK
1. Extract all discrete, verifiable factual claims from each document.
2. Compare every claim against every other claim from a different document.
3. Identify pairs that are logically contradictory.
4. Classify each contradiction by type: `factual` (direct fact dispute), `numerical` (conflicting numbers), `temporal` (date/sequence conflict), `categorical` (mutually exclusive classifications), or `definitional` (conflicting definitions of a term).
5. Assess the severity of each conflict: `critical` (impacts a core decision or regulatory requirement), `major` (significant disagreement on an important point), `minor` (disagreement on a peripheral detail), or `informational` (notable but low impact).

## CONSTRAINTS
- Do not flag differences in phrasing or emphasis as contradictions unless the underlying facts are mutually exclusive.
- If a claim in Document A is more specific than a claim in Document B, and the specific claim does not logically contradict the general one, do not flag it.
- Ignore contradictions that are resolved by a clear temporal precedence rule (e.g., Document B is an explicit update to Document A) unless the [TEMPORAL_PRECEDENCE_RULES] field specifies otherwise.
- If [RISK_LEVEL] is `high`, you must flag borderline cases and include a `confidence` score (0.0 to 1.0) for each contradiction.

## OUTPUT_SCHEMA
Return a valid JSON object with the following structure:
{
  "contradictions": [
    {
      "id": "string, unique contradiction identifier",
      "claim_a": {
        "document_id": "string",
        "excerpt": "string, the exact text of the claim",
        "paraphrase": "string, a normalized version of the claim"
      },
      "claim_b": {
        "document_id": "string",
        "excerpt": "string",
        "paraphrase": "string"
      },
      "conflict_type": "factual | numerical | temporal | categorical | definitional",
      "severity": "critical | major | minor | informational",
      "explanation": "string, a concise explanation of why these claims cannot both be true",
      "confidence": number, // only if RISK_LEVEL is high
      "resolution_hint": "string or null, a suggestion for how a human might resolve the conflict"
    }
  ],
  "analysis_metadata": {
    "total_documents_analyzed": number,
    "total_claims_extracted": number,
    "total_contradictions_found": number,
    "execution_notes": "string, any caveats about the analysis"
  }
}

## EXAMPLES
[EXAMPLES]

## TEMPORAL PRECEDENCE RULES
[TEMPORAL_PRECEDENCE_RULES]

## RISK_LEVEL
[RISK_LEVEL]

To adapt this template, start by populating the [DOCUMENTS] placeholder with your retrieved chunks, ensuring each has a stable id that maps back to your source system. The [EXAMPLES] field is critical for aligning the model with your definition of contradiction; include at least two positive examples (clear contradictions with correct classifications) and one negative example (similar claims that are not contradictory). Set [RISK_LEVEL] to high for regulated domains like healthcare or finance to enable confidence scoring and force the model to surface borderline cases for human review. The [TEMPORAL_PRECEDENCE_RULES] field should be a short natural-language policy, such as "Documents with a later date field supersede earlier documents on the same fact." After generating the output, always validate the JSON structure and run the eval checks described in the Testing and Evaluation section of this playbook before integrating the results into a downstream workflow.

The Multi-Source Conflict Detection prompt is designed to operate as a post-retrieval, pre-generation step in a RAG pipeline. After your retriever returns a set of candidate documents for a user query, batch those documents into this prompt to produce a structured conflict report. The report then becomes a decision point: if no conflicts are found, proceed to answer generation; if conflicts are detected, route the output to a conflict-aware answer prompt or escalate to a human review queue. This prompt is not a user-facing answer generator—it is an analysis tool that feeds downstream logic.

Wire this prompt into your application as a dedicated service with a strict JSON output contract. Use a model that supports structured outputs (e.g., GPT-4o with response_format or Claude with tool use) and set the [OUTPUT_SCHEMA] to match your internal conflict report schema. Implement a validation layer that checks for required fields (source_pairs, claim_excerpts, conflict_type, severity) and rejects malformed responses. On validation failure, retry once with the error message appended to the prompt. Log every conflict detection call with the input document count, detected conflict count, and validation status for observability. For high-stakes domains, add a human review step when severity is critical or when the number of detected conflicts exceeds a configurable threshold.

Common failure modes include false positives from paraphrased agreement (the model flags two sources saying the same thing differently as a conflict) and missed contradictions when documents use different terminology for the same entity. Mitigate the first by tuning the [CONSTRAINTS] field to require semantic equivalence checks before flagging a conflict. Mitigate the second by ensuring your retrieval step includes query expansion and that documents are pre-processed with entity resolution. Build an eval set with known conflict and no-conflict document pairs, and measure precision and recall against human labels before deploying. Start with a low severity threshold for human review and tighten it as your eval metrics improve.