RAG Source Conflict Clarification Prompt

Use this prompt when your grounded QA system retrieves multiple passages that directly contradict each other on a material fact, and you need the model to surface the conflict to the user rather than silently picking a side or hallucinating a compromise. The ideal user is an AI engineer or product developer building a customer-facing assistant, internal research tool, or support copilot where source fidelity is a hard requirement. The job-to-be-done is not just answering a question—it is maintaining user trust by making retrieval conflicts transparent and actionable. This prompt is designed for production RAG pipelines where the retrieved context is already in the prompt window, and the model's task is to analyze, cite, and clarify, not to perform the retrieval itself.

This prompt is not appropriate when the retrieved passages are merely different in emphasis or style rather than factually contradictory. Do not use it for low-stakes creative generation, summarization of opinion pieces, or scenarios where the user expects a single synthesized answer without source interrogation. It is also the wrong tool when the contradiction stems from a broken retrieval step—if your vector search is returning irrelevant chunks, fix retrieval before adding a conflict resolution layer. The prompt assumes that the input context contains at least two passages with identifiable, citable contradictions. If your system cannot reliably surface conflicting passages, you will get false-positive conflict reports that erode user confidence.

Before deploying this prompt, ensure your evaluation harness includes checks for source attribution accuracy, conflict framing neutrality, and clarification question specificity. A common failure mode is the model inventing a conflict where passages are actually complementary, or framing one source as 'correct' and the other as 'wrong' without evidence. Wire the prompt into an application flow that logs conflict instances, tracks whether users respond to clarification questions, and measures whether resolution rates improve over time. If the domain is regulated—such as healthcare, legal, or financial advice—always route the final answer through human review and never let the model resolve the conflict autonomously. Start with a high retrieval similarity threshold to surface only clear contradictions, then tune downward based on production data.

This prompt template is designed to be dropped into a RAG pipeline immediately after the retrieval step. It instructs the model to act as a neutral arbiter when presented with conflicting source passages. Instead of hallucinating a resolution or arbitrarily picking a winner, the model must produce a structured conflict report that cites the disagreeing sources verbatim and then formulate a single, precise clarification question. The goal is to surface the contradiction to the user transparently and gather the minimum information needed to resolve it, preserving user trust and avoiding wasted turns.

codeCopy
You are a precise research assistant. Your task is to analyze the provided source passages and the user's question. If the sources contradict each other on a material fact, you must not answer the user's question directly. Instead, you must produce a conflict summary and a single clarification question.

USER QUESTION:
[USER_QUESTION]

RETRIEVED SOURCES:
[RETRIEVED_SOURCES]

INSTRUCTIONS:
1.  **Analyze:** Determine if the sources contain a direct contradiction on a fact relevant to the user's question. A contradiction exists only if two or more sources make mutually exclusive claims.
2.  **If No Contradiction:** If the sources are consistent or address different aspects, output a JSON object with a single key "status" and the value "no_conflict". Do not answer the question.
3.  **If Contradiction Exists:** Output a JSON object with the following structure:
    *   `status`: "conflict"
    *   `conflict_summary`: A concise, neutral summary of the point of disagreement. Frame it as "Source A states X, while Source B states Y." Do not infer which source is correct.
    *   `source_a_claim`: A direct quote from the first source that supports its claim.
    *   `source_b_claim`: A direct quote from the second source that supports its contradictory claim.
    *   `clarification_question`: A single, targeted question for the user that would resolve the contradiction. The question must be answerable without the user needing to see the original sources. Do not ask "Which source is correct?"

CONSTRAINTS:
*   Never fabricate a resolution. Your only job is to identify and report the conflict.
*   Cite sources exactly. Do not paraphrase the conflicting claims.
*   The clarification question must be specific to the factual disagreement.
*   Output only the valid JSON object. No other text.

OUTPUT SCHEMA:
{
  "status": "conflict",
  "conflict_summary": "string",
  "source_a_claim": "string",
  "source_b_claim": "string",
  "clarification_question": "string"
}

To adapt this template, replace the [USER_QUESTION] and [RETRIEVED_SOURCES] placeholders with your application's runtime variables. The [RETRIEVED_SOURCES] should be a pre-formatted string containing the text of each passage, clearly delimited with identifiers (e.g., [Source 1] ... [Source 2] ...). This prompt is designed for a single LLM call. For high-stakes applications, you should validate the output JSON against the schema and check that the source_a_claim and source_b_claim strings are exact substrings of the [RETRIEVED_SOURCES] input. If validation fails, a retry loop with a failure reason appended to the prompt is a standard recovery pattern. Always log the raw prompt, the model's output, and the validation result to monitor the frequency and nature of source conflicts in your system.

This prompt is designed to sit inside a retrieval-augmented generation pipeline after the retrieval step but before the final answer synthesis. When your retriever returns multiple passages with contradictory claims, the prompt produces a structured conflict summary and a targeted clarification question instead of letting the model silently pick a side or hallucinate a reconciliation. The implementation harness must catch the conflict signal early, route it to the user or a review queue, and prevent the system from proceeding to answer generation until the conflict is resolved.

Wire the prompt as a decision gate in your RAG pipeline. After retrieval, run a lightweight contradiction detector (e.g., an NLI model or a keyword-based overlap check) to flag candidate conflicts. If flagged, invoke this prompt with the conflicting passages and the user's original query. Parse the output against a strict schema: expect fields like conflict_summary, source_a_citation, source_b_citation, disagreeing_claim, and clarification_question. Validate that every citation maps back to a real retrieved passage ID. If the output fails schema validation or contains citations to non-existent sources, retry once with a stricter constraint message. After two failures, escalate to a human reviewer with the raw passages and the failed outputs logged. For high-stakes domains (healthcare, legal, finance), always require human approval before surfacing the clarification question to the end user.

Model choice matters here. Use a model with strong instruction-following and citation discipline (e.g., GPT-4o, Claude 3.5 Sonnet) rather than a smaller or faster model that may drop citations or invent source mappings. Set temperature low (0.0–0.2) to reduce creative paraphrasing of the conflict. Log every invocation: the retrieved passage IDs, the conflict detection score, the raw prompt, the model output, the validation result, and whether the clarification was shown to the user or escalated. This audit trail is essential for debugging false-positive conflict flags and for demonstrating source-grounded behavior to compliance reviewers. Avoid the temptation to skip the conflict gate for latency reasons—surfacing a contradictory answer is far more expensive than a single clarification turn.