Contradictory Evidence Abstention Trigger Prompt

This prompt is designed for RAG system operators who manage knowledge bases where source documents can disagree. When a user query hits multiple retrieved passages that contradict each other, the system must not silently pick a winner or blend conflicting facts into a fluent but misleading answer. Instead, this prompt produces a structured conflict detection signal and triggers abstention when contradictions exceed a configurable threshold. Use this prompt in high-stakes domains such as legal research, clinical guidelines, financial compliance, and policy documentation where surfacing conflict is safer than synthesizing a false consensus.

This prompt belongs in the pre-generation gate layer of a RAG pipeline, after retrieval and before answer synthesis. It assumes you have already retrieved multiple evidence passages and need to decide whether the evidence is consistent enough to proceed. The prompt expects a set of retrieved passages and a user query as input, and it outputs a structured decision: either a proceed signal with a conflict score below threshold, or an abstain signal with a detailed conflict report. The conflict threshold is a configurable parameter you set based on your domain's risk tolerance—legal applications might set a near-zero threshold, while internal research tools might allow moderate conflict with explicit caveats.

Do not use this prompt when you have only a single retrieved passage, when the knowledge base is curated to be internally consistent by design, or when the cost of abstention is higher than the cost of a qualified answer. In low-stakes exploratory search or creative ideation workflows, a simpler confidence score or sufficiency check is often more appropriate. Also avoid this prompt when your retrieval pipeline already performs authoritative source selection—if you have a single canonical source per query, conflict detection adds latency without benefit. For systems that need a binary answer/no-answer gate without detailed conflict analysis, start with the RAG Abstention Decision Prompt Template instead.

Before deploying this prompt, you must calibrate your conflict threshold against a labeled evaluation dataset containing known contradictory passage sets. Run the prompt against cases where sources genuinely conflict, where they appear to conflict due to temporal or contextual differences, and where they are fully consistent. Measure both the missed-conflict rate (system proceeds when it should abstain) and the false-alarm rate (system abstains when evidence is actually reconcilable). These metrics will determine whether the prompt's conflict detection logic is tuned correctly for your domain. In regulated environments, always log the full conflict report alongside the abstention decision for audit review.

This prompt is designed to be pasted directly into your system prompt or as a pre-generation instruction block. It instructs the model to analyze all provided source passages, identify factual conflicts, and produce a structured abstention signal when contradictions exceed a defined severity or count threshold. The goal is to prevent the model from silently choosing one source over another or synthesizing a misleading compromise.

textCopy
You are a contradiction detection system operating over retrieved evidence. Your task is to analyze the provided source passages and determine whether they contain material factual contradictions that prevent a reliable answer.

## INPUT
[QUERY]

## SOURCE PASSAGES
[SOURCES]

## CONTRADICTION THRESHOLD
- Minimum conflicting source pairs to trigger abstention: [CONFLICT_PAIR_THRESHOLD]
- Minimum severity level to count a conflict: [SEVERITY_THRESHOLD]

## INSTRUCTIONS
1. Extract all factual claims from each source passage.
2. Identify pairs of claims that directly contradict each other. A contradiction exists when two claims cannot both be true.
3. For each contradiction, assign a severity level:
   - **CRITICAL**: The contradiction involves a core fact that would change the answer.
   - **MATERIAL**: The contradiction involves a supporting fact that affects answer quality.
   - **MINOR**: The contradiction involves peripheral details that do not affect the main answer.
4. Count the number of conflicting source pairs at or above the severity threshold.
5. If the count meets or exceeds the conflict pair threshold, abstain from answering.
6. If abstaining, output only the conflict detection payload. Do not provide an answer.
7. If not abstaining, synthesize an answer that explicitly acknowledges remaining conflicts and attributes each position to its source.

## OUTPUT SCHEMA
```json
{
  "abstain": boolean,
  "conflict_pairs_found": number,
  "conflicts": [
    {
      "claim_a": string,
      "source_a": string,
      "claim_b": string,
      "source_b": string,
      "severity": "CRITICAL" | "MATERIAL" | "MINOR",
      "explanation": string
    }
  ],
  "answer": string | null
}

CONSTRAINTS

• Do not fabricate conflicts where sources use different terminology for the same fact.
• Do not treat missing information in one source as a contradiction.
• If sources address different time periods or scopes, they may not conflict.
• When abstaining, the answer field must be null.

To adapt this template, adjust the [CONFLICT_PAIR_THRESHOLD] and [SEVERITY_THRESHOLD] based on your risk tolerance. For high-stakes domains like clinical or legal RAG, set the severity threshold to MATERIAL and the pair threshold to 1 so that any meaningful conflict triggers abstention. For lower-stakes applications, you might set the severity threshold to CRITICAL and the pair threshold to 3 to allow minor disagreements through. The [SOURCES] placeholder should receive your retrieved passages with clear source identifiers so the model can attribute conflicts precisely. Always validate the output against the JSON schema before routing the abstention decision to your application logic.

This prompt is not a standalone answer generator; it is a pre-generation gate that sits between retrieval and synthesis. In a production RAG pipeline, the retrieved context and user query are passed to this prompt first. The model returns a structured conflict signal. If the abstain flag is true, the system must halt answer generation and instead surface the conflict to the user or escalate to a human review queue. If abstain is false, the pipeline proceeds to the answer synthesis prompt, optionally passing the conflict_notes field as additional context to guide cautious wording.

Validation and retry logic is critical because a malformed JSON response from this prompt can crash the downstream pipeline. Implement a JSON schema validator that checks for the required fields: abstain (boolean), conflict_detected (boolean), conflict_count (integer), conflict_notes (string), and conflicting_sources (array of objects with source_id and claim). If validation fails, retry up to two times with a repair prompt that includes the raw output and the schema. If retries are exhausted, default to abstain: true and log the failure as a safety-override event. For high-stakes domains like healthcare or legal, route all abstain: true decisions to a human review queue with the full evidence packet attached.

Model choice and latency matter here. This prompt performs a reasoning task—comparing claims across sources—so it benefits from a capable model like GPT-4o or Claude 3.5 Sonnet. However, it adds latency before answer generation. To keep response times acceptable, run this prompt in parallel with a lightweight answerability classifier. If the classifier returns low answerability, skip the conflict check and abstain immediately. Log every decision—the query, retrieved source IDs, conflict signal, and final action—to build an eval dataset for tuning the abstention threshold over time. Avoid using this prompt on every query if your retrieval pipeline rarely returns conflicting sources; instead, trigger it only when source count exceeds a threshold or when source metadata indicates different authors, dates, or document types.