Evidence Reconciliation Prompt for Contradictory Sources

This prompt is for knowledge-base teams and RAG application developers who need to systematically reconcile factual disagreements across multiple source documents. The core job-to-be-done is producing a structured reconciliation table that maps claims to their supporting sources, identifies which claims are disputed, and flags irreconcilable differences without fabricating false consensus. The ideal user is an engineer or product manager building a decision-support system, research assistant, or compliance review tool where users must see contradictions rather than have them silently resolved.

Use this prompt when you have a retrieval set containing sources that directly contradict each other on material facts—for example, conflicting financial figures in two earnings reports, contradictory clinical trial outcomes, or opposing technical specifications in vendor documentation. The prompt requires pre-retrieved source passages as input, along with a clear specification of the factual dimensions to reconcile. It is not suitable for single-source analysis, general summarization, or scenarios where sources agree. Do not use this prompt when the contradictions are matters of opinion rather than verifiable fact, or when the sources are not authoritative enough to warrant structured reconciliation. The output is a machine-readable reconciliation table, not a narrative answer, making it appropriate for downstream processing, audit trails, and user-facing comparison UIs.

Before deploying this prompt, ensure you have a retrieval pipeline that can surface contradictory passages, not just top-k relevant results. The prompt assumes you have already identified that disagreement exists; it does not perform contradiction detection from scratch. Pair this prompt with an evidence conflict detection step upstream and a human review queue for irreconcilable flags. In regulated domains such as healthcare, finance, or legal, always route outputs flagged as irreconcilable to a subject-matter expert before they reach end users. The reconciliation table should be treated as an intermediate artifact, not a final answer.

The prompt below is designed to be copied directly into your application or evaluation harness. It instructs the model to produce a structured reconciliation table that maps claims to their supporting and contradicting sources, identifies disputed facts, and flags irreconcilable differences. Every placeholder is wrapped in square brackets and must be replaced with concrete values before the prompt is sent to the model. The template assumes you have already retrieved a set of potentially contradictory passages and need the model to surface disagreements rather than silently harmonize them.

textCopy
You are an evidence reconciliation analyst. Your task is to examine a set of source passages that may contain contradictory claims and produce a structured reconciliation report.

## INPUT SOURCES
[SOURCES]

## CLAIMS TO VERIFY
[CLAIMS]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "reconciliation_table": [
    {
      "claim": "string (the claim being examined)",
      "supporting_sources": [
        {
          "source_id": "string",
          "excerpt": "string (the specific text that supports the claim)",
          "support_strength": "direct | indirect | weak"
        }
      ],
      "contradicting_sources": [
        {
          "source_id": "string",
          "excerpt": "string (the specific text that contradicts the claim)",
          "contradiction_type": "direct_negation | alternative_fact | scope_disagreement | framing_disagreement"
        }
      ],
      "disputed": true | false,
      "resolution_notes": "string (explain whether the contradiction can be resolved by scope, recency, authority, or context, or if it is irreconcilable)"
    }
  ],
  "irreconcilable_differences": [
    {
      "claim": "string",
      "source_a_id": "string",
      "source_a_position": "string",
      "source_b_id": "string",
      "source_b_position": "string",
      "reason_irreconcilable": "string"
    }
  ],
  "overall_assessment": "string (summary of the evidence landscape, degree of consensus, and confidence in any conclusions)"
}

## CONSTRAINTS
- Do not fabricate consensus where sources genuinely disagree.
- If a claim has no supporting or contradicting evidence in the provided sources, mark it as disputed: false with empty arrays and note the absence of evidence in resolution_notes.
- Preserve the exact wording of excerpts; do not paraphrase.
- If sources disagree on scope, recency, or framing rather than direct facts, capture that in contradiction_type and resolution_notes.
- Do not introduce external knowledge. Work only with the provided [SOURCES].
- If [RISK_LEVEL] is "high", append a human_review_required: true flag to the output and include a summary of what a reviewer should verify.

## RISK LEVEL
[RISK_LEVEL]

## EXAMPLES
[EXAMPLES]

To adapt this template, start by replacing [SOURCES] with your retrieved passages, each tagged with a unique source_id and the full passage text. Replace [CLAIMS] with the specific factual assertions you need to verify across those sources—these may come from a prior extraction step or from user-submitted statements. Set [RISK_LEVEL] to "high" for regulated, safety-critical, or customer-facing workflows to force a human-review flag in the output. The [EXAMPLES] placeholder should contain one or two worked examples showing the expected reconciliation behavior, especially for cases where sources genuinely conflict. After generating the output, validate the JSON structure against the schema, check that every claim in the input appears in the reconciliation_table, and verify that no source_id references are hallucinated. For high-risk deployments, route outputs with irreconcilable_differences to a human reviewer before any downstream consumption.

The Evidence Reconciliation Prompt is designed as a structured analysis step within a larger RAG or document-intelligence pipeline, not as a standalone chat interaction. It expects pre-retrieved source passages that have already been identified as potentially contradictory. The application layer is responsible for assembling the [SOURCES] array, injecting the [CLAIM_OR_QUESTION], and parsing the reconciliation table output. This prompt works best when it follows an upstream conflict detection step that flags which passages disagree, so the reconciliation prompt receives a focused set of contradictory sources rather than an undifferentiated retrieval dump.

Input assembly requires mapping each source to a structured object with source_id, source_label, passage_text, and optional metadata fields (date, authority score, retrieval rank). The application should deduplicate near-identical passages before calling this prompt to avoid wasting tokens on redundant evidence. Model choice matters: use a model with strong instruction-following and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or Gemini 2.0 Flash with JSON mode enabled). For high-stakes domains (legal, medical, financial), prefer models with lower hallucination rates on structured extraction tasks and always enable temperature=0 or a low value (0.1–0.2) to maximize output stability. Tool use is not required for this prompt—it is a pure text-in, structured-text-out operation—but you may wire it as a tool if your agent framework requires tool-call formatting.

Output validation is critical because the reconciliation table drives downstream decisions. Implement a post-generation validator that checks: (1) every source_id in the output exists in the input source list, (2) no source is omitted from the reconciliation table unless explicitly marked as irrelevant with a reason, (3) the reconciliation_status field uses only the allowed enum values (confirmed, disputed, irreconcilable, insufficient_evidence), (4) every disputed or irreconcilable claim includes specific evidence excerpts from both sides, and (5) the output is valid JSON matching the expected schema. If validation fails, retry once with the validation errors appended to the prompt as [PREVIOUS_VALIDATION_ERRORS]. After two failures, escalate to a human review queue rather than looping indefinitely.

Logging and observability should capture: the input source count, the number of claims analyzed, the distribution of reconciliation statuses, validation pass/fail, retry count, and latency. Store the full prompt and response for audit trails, especially in regulated workflows. Human review gates are required when: (1) the prompt flags claims as irreconcilable, (2) the confidence score for any claim falls below a configurable threshold (e.g., <0.7), (3) the output contains insufficient_evidence for a claim the user needs resolved, or (4) the validator rejects the output twice. The review interface should display the original sources side-by-side with the reconciliation table, highlighting disputed claims for rapid human judgment.

Integration patterns: wire this prompt as a synchronous step in a request-response pipeline for interactive applications (latency target <5 seconds), or as an asynchronous batch job for document review workflows. For RAG systems, place this prompt after retrieval and before answer generation—the reconciliation table informs the final answer prompt about which claims are settled versus disputed. Avoid using this prompt on sources that are not genuinely contradictory; it will fabricate disagreements to satisfy the task framing. Pre-filter with a lightweight contradiction detection prompt or embedding similarity threshold before invoking reconciliation. Do not skip the human review gate for irreconcilable claims in customer-facing or decision-support applications—presenting unresolved contradictions as settled facts causes more harm than admitting uncertainty.