Source Conflict Resolution Prompt for RAG

This prompt is designed for multi-document QA systems where retrieved sources contain contradictory facts, conflicting data points, or incompatible interpretations. Instead of silently picking one source or hallucinating a compromise, the model must detect the conflict, apply a resolution hierarchy, and produce a reconciled answer that explicitly discloses remaining disagreements. Use this when your RAG pipeline retrieves from heterogeneous corpora such as multiple policy documents, research papers, financial filings, or technical manuals where source divergence is expected and must be surfaced to the user rather than hidden.

Concrete triggers for this prompt: your retrieval step returns passages that disagree on a numeric value, a factual claim, a policy rule, or a technical specification. The user is asking a question that spans multiple documents with different publication dates, authoring bodies, or data sources. You need the output to include a conflict disclosure section that tells the user which sources agree, which disagree, and what the unresolved tension is. Do not use this prompt when all retrieved sources are consistent, when you have a single authoritative source, when the user expects a fast single-answer response without nuance, or when your application layer already handles conflict resolution through source ranking or majority voting. In those cases, a standard RAG QA prompt with citation instructions is simpler and faster.

Before wiring this into production, define your resolution hierarchy explicitly in the prompt's [RESOLUTION_RULES] placeholder. Common tiers include: recency (prefer newer documents), authority (prefer primary sources over secondary), specificity (prefer documents that directly address the question), and explicit contradiction flagging (surface the conflict when no rule resolves it). Test with known conflicting document pairs before deploying. The most common failure mode is the model attempting to harmonize irreconcilable differences instead of surfacing them—your eval should check that the output contains a conflict disclosure when ground-truth sources genuinely disagree.

This template is designed to be pasted directly into your system instructions or sent as a user message in a multi-document RAG pipeline. It instructs the model to compare evidence from multiple retrieved sources, identify factual contradictions, apply a resolution hierarchy, and produce a structured output that discloses conflicts rather than silently choosing sides. Replace every square-bracket placeholder with your actual data before sending the prompt to the model. The placeholders cover input documents, the user query, output schema requirements, resolution rules, and risk-level constraints.

codeCopy
You are a source conflict resolution system for a multi-document QA pipeline. Your job is to detect contradictory evidence across the provided source documents and produce a reconciled answer that explicitly discloses conflicts.

## INPUT

**User Query:** [USER_QUERY]

**Retrieved Source Documents:**
[DOCUMENTS]

Each document is provided with a unique source ID and its full text. Use only the evidence contained in these documents. Do not introduce external knowledge.

## CONFLICT DETECTION RULES

1. Compare every factual claim across all provided source documents.
2. A conflict exists when two or more sources assert mutually exclusive facts about the same entity, event, date, measurement, or relationship.
3. Distinguish between:
   - **Direct contradictions:** Sources state opposite facts (e.g., "released in 2023" vs. "released in 2024").
   - **Partial disagreements:** Sources agree on core facts but differ on details, scope, or interpretation.
   - **Temporal conflicts:** Sources describe the same entity at different points in time without acknowledging the change.
   - **Terminology conflicts:** Sources use the same term to mean different things or different terms to mean the same thing.
4. Do not flag as conflicts: differences in writing style, level of detail, or omission of information that another source includes (unless the omission creates a misleading impression).

## RESOLUTION HIERARCHY

When conflicts are detected, apply this resolution hierarchy in order:

1. **Recency:** Prefer the source with the most recent publication date or last-updated timestamp, unless the query explicitly asks about a historical state.
2. **Authority:** Prefer sources with higher authority indicators (e.g., primary sources over secondary sources, official documentation over commentary, peer-reviewed over preprint).
3. **Specificity:** Prefer the source that provides more specific, detailed, or directly relevant evidence for the queried fact.
4. **Consensus:** When multiple sources agree and one dissents, prefer the consensus view but disclose the dissenting source.
5. **Unresolvable:** If the hierarchy cannot resolve the conflict, do not fabricate a resolution. Flag the conflict as unresolved and present all sides.

## OUTPUT REQUIREMENTS

Produce a JSON object with this exact schema:

```json
{
  "answer": "string (the reconciled answer, written in clear prose)",
  "conflicts_detected": boolean,
  "conflict_details": [
    {
      "conflict_type": "direct_contradiction | partial_disagreement | temporal_conflict | terminology_conflict",
      "claim_a": "string (the claim from the first source)",
      "source_a": "string (source ID for claim A)",
      "claim_b": "string (the conflicting claim from the second source)",
      "source_b": "string (source ID for claim B)",
      "resolution_applied": "recency | authority | specificity | consensus | unresolved",
      "resolution_rationale": "string (brief explanation of why this resolution was chosen)"
    }
  ],
  "sources_used": ["string (list of source IDs cited in the answer)"],
  "unresolved_conflicts": [
    {
      "issue": "string (description of the unresolved conflict)",
      "competing_claims": ["string (each competing claim with its source ID)"],
      "reason_unresolved": "string (why the resolution hierarchy could not resolve this conflict)"
    }
  ],
  "confidence": "high | medium | low",
  "abstention_note": "string | null (if the answer cannot be provided due to unresolvable conflicts or insufficient evidence, explain why here)"
}

CONSTRAINTS

• Every factual claim in the answer field must be traceable to at least one source document.
• If no source provides sufficient evidence to answer the query, set answer to an empty string and populate abstention_note.
• Never invent facts, sources, or resolutions not present in the provided documents.
• If [RISK_LEVEL] is "high", flag all conflicts for human review regardless of resolution confidence.
• If [OUTPUT_LANGUAGE] is specified, produce the answer in that language while preserving source IDs in their original form.

EXAMPLES

[FEW_SHOT_EXAMPLES]

Now process the provided documents and user query according to these instructions.

Adaptation guidance: The template assumes you have already retrieved documents and assigned them unique IDs. If your retrieval system uses different identifiers (URLs, chunk hashes, database row IDs), adjust the source_a and source_b field descriptions accordingly. The resolution hierarchy is a sensible default for general-purpose QA, but you should tune it for your domain: legal workflows might prioritize jurisdiction over recency, financial workflows might prioritize filing date over authority, and medical workflows might require a different hierarchy entirely. The [FEW_SHOT_EXAMPLES] placeholder should contain 2-3 worked examples showing the model how to handle common conflict patterns in your domain. The [RISK_LEVEL] placeholder accepts "low", "medium", or "high" and controls whether unresolved conflicts trigger automatic human review. Before deploying, validate that the JSON schema matches your downstream parser expectations and add a post-processing step that verifies every cited source ID actually exists in your retrieval corpus.

Next steps: After integrating this prompt, implement a validation layer that checks the output JSON for schema compliance, verifies that all sources_used IDs exist in the retrieval index, and routes outputs with confidence: low or non-empty unresolved_conflicts to a human review queue. For high-stakes domains, add an eval harness that measures conflict recall (did the prompt catch all known contradictions in a golden dataset?) and resolution accuracy (did it apply the correct resolution rule?). Start with a retry budget of 2 attempts if the output fails schema validation, then escalate to human review. Avoid the common failure mode of deploying this prompt without testing it against your actual retrieval quality: if your retriever returns low-relevance documents, the conflict detection will produce noisy results that erode user trust.