Contradiction Resolution Prompt for Multi-Hop Evidence

This prompt is designed for operators of high-stakes Retrieval-Augmented Generation (RAG) systems in domains like legal, finance, and compliance, where the cost of a false consensus is high. Its core job is to handle a specific failure mode: when two or more pieces of retrieved evidence directly contradict each other. Instead of forcing the model to synthesize a single, fluent, but misleading answer, this prompt instructs it to produce a structured 'Conflict Report.' This report transparently analyzes the nature of the contradiction, assesses the relative credibility of the conflicting sources, and recommends a resolution strategy for the user. The primary user is an AI engineer or product manager who needs their system to surface ambiguity rather than hide it.

The ideal scenario for this prompt is a multi-hop reasoning task where the final answer depends on reconciling facts from multiple documents. For example, a financial analyst might ask, 'What was the company's reported revenue in Q3?' and the system retrieves a press release stating $1.2B and a later SEC filing stating $1.15B. A standard prompt might arbitrarily pick one or create a confusing blend. This prompt, however, will produce a report identifying the source of each figure, noting the temporal and credibility differences (e.g., 'preliminary release' vs. 'official filing'), and recommending the audited filing as the authoritative source. It is a critical component of a trustworthy AI architecture where auditability is a non-negotiable requirement.

Do not use this prompt when you need a single, definitive answer for a low-stakes application, when all retrieved sources are already consistent, or when the contradiction is trivial and does not affect the user's decision. Applying this prompt to a simple factual query with no conflict will result in a verbose and unnecessary report, degrading the user experience. It is also not a substitute for a robust retrieval pipeline; if your system frequently retrieves contradictory information due to poor chunking or search quality, fix the retrieval step first. The next step after understanding this use case is to review the 'Prompt Template' section to see the exact instructions and adapt the placeholders to your specific source metadata.

This prompt template is designed to be dropped directly into your application's system prompt or a high-priority user message. It instructs the model to act as a neutral analyst, not an answer-generator. The core job is to surface and structure a contradiction between two or more pieces of evidence, assess the credibility of each source, and recommend a resolution strategy without fabricating a consensus. The template uses square-bracket placeholders for all dynamic inputs, making it safe to wire into a production RAG pipeline where evidence chunks and user queries are injected programmatically.

textCopy
You are a neutral evidence analyst. Your task is to resolve contradictions in provided evidence, not to answer the user's question directly.

## User Query
[USER_QUERY]

## Contradictory Evidence
### Evidence A
Source: [SOURCE_A_METADATA]
Text: [EVIDENCE_A_TEXT]

### Evidence B
Source: [SOURCE_B_METADATA]
Text: [EVIDENCE_B_TEXT]

[ADDITIONAL_EVIDENCE_BLOCKS]

## Instructions
1.  **Identify the Core Conflict:** State the exact point of contradiction between the evidence pieces in one clear sentence.
2.  **Assess Source Credibility:** For each source, note factors that affect its reliability (e.g., publication date, author expertise, document type, potential biases evident in the text). Do not assume one source is correct.
3.  **Analyze the Conflict:** Explain possible reasons for the contradiction (e.g., temporal change, different definitions, factual error, incomplete context).
4.  **Recommend a Resolution Strategy:** Choose exactly one of the following strategies and justify your choice:
    - **Prefer A:** Evidence A is more credible/relevant.
    - **Prefer B:** Evidence B is more credible/relevant.
    - **Reconcile:** The contradiction is apparent and can be resolved by clarifying context or definitions.
    - **Abstain:** The conflict cannot be resolved with the available evidence.
5.  **Generate a Conflict Report:** Structure your entire output as a JSON object matching the schema below.

## Output Schema
{
  "core_conflict": "string",
  "source_assessments": [
    {
      "source_label": "A",
      "credibility_factors": ["string"],
      "reliability_notes": "string"
    },
    {
      "source_label": "B",
      "credibility_factors": ["string"],
      "reliability_notes": "string"
    }
  ],
  "conflict_analysis": "string",
  "resolution_strategy": "Prefer A" | "Prefer B" | "Reconcile" | "Abstain",
  "strategy_justification": "string",
  "synthesized_answer": "string or null (provide a concise answer only if strategy is Prefer A, Prefer B, or Reconcile, grounded strictly in the chosen evidence)"
}

## Constraints
- Do not invent information to resolve the conflict.
- If you abstain, set `synthesized_answer` to null.
- Your `conflict_analysis` must fairly represent both sides.

To adapt this template, replace the placeholders with your application's data. [USER_QUERY] is the original question that triggered the retrieval. [EVIDENCE_A_TEXT] and [EVIDENCE_B_TEXT] are the raw text chunks from your vector store or search index. Crucially, [SOURCE_A_METADATA] should be a pre-formatted string containing the document title, date, author, and any other provenance information you have—this is what the model uses for credibility assessment. If you have more than two conflicting pieces of evidence, duplicate the evidence block structure and add them to the [ADDITIONAL_EVIDENCE_BLOCKS] section. The output schema is strict JSON, so you must use a model that supports structured output or a robust JSON parser in your harness. For high-stakes domains like legal or financial analysis, always route the final conflict_analysis and strategy_justification fields to a human review queue before the synthesized_answer is shown to an end user.

The Contradiction Resolution Prompt is not a fire-and-forget answer generator. It is a conflict analysis engine that must be embedded in a harness that validates its output, enforces its abstention from forced resolution, and routes high-risk conflicts to human review. The prompt expects at least two contradictory evidence passages, optional source metadata, and a domain context like legal or finance. The harness should assemble these inputs from your retrieval pipeline, ensuring that the passages passed to the prompt genuinely conflict rather than simply address different aspects of the same question. A pre-filter that detects semantic contradiction before invoking this prompt will reduce unnecessary LLM calls and prevent the model from manufacturing conflicts where none exist.

Wire the prompt into an application workflow with the following guardrails. Input assembly: retrieve candidate passages, run a lightweight contradiction detector (e.g., an NLI model or a smaller LLM call), and only invoke this prompt when a conflict score exceeds a threshold. Output validation: parse the JSON response and enforce the schema—conflict_type, source_credibility_assessment, resolution_strategies, and recommended_strategy must all be present. Reject outputs that pick a winner without analysis or that omit a strategy. Retry logic: if validation fails, retry once with a stricter system instruction that forbids premature resolution. If the retry also fails, log the raw output and escalate. Human review gate: for conflict_type values of direct_contradiction or methodological_conflict in regulated domains, route the conflict report to a review queue before any downstream action is taken. Model choice: use a model with strong reasoning and instruction-following, such as Claude 3.5 Sonnet or GPT-4o. Avoid smaller models that may collapse the conflict into a false consensus. Logging: capture the full prompt, response, validation result, and reviewer decision for audit trails, especially in legal and finance use cases where conflict handling decisions must be defensible.

Do not use this prompt inside an autonomous agent loop that automatically acts on the resolution recommendation. The recommended strategy is advisory, not executable. Downstream actions—such as preferring one source, requesting additional evidence, or presenting both sides to the user—must be implemented in application code with explicit policy rules. The prompt's value is in producing a structured, fair analysis of the conflict, not in making the final call. Test the harness end-to-end with known contradictory document pairs and verify that the system never silently adopts one source without a logged conflict report and, where required, human approval.