Evidence Dispute Summary Prompt Template

This prompt is built for a specific moment in an evidence pipeline: a conflict has already been detected, the relevant passages have been extracted, and now someone needs a structured review packet before making a decision. The primary user is an operations team member or a reviewer in a queue who must understand what the dispute is about, which sources disagree, how reliable each source appears, and what the recommended next step should be. The prompt does not detect conflicts, rank evidence, or decide the final answer. It summarizes a known dispute so a human can act on it efficiently.

Use this prompt when your system has already run a conflict detection step—such as the Multi-Source Conflict Detection Prompt Template or the Contradiction Extraction Prompt for Research Briefs—and you have a set of opposing source positions ready. The prompt expects a contested claim and at least two conflicting source passages as input. It produces a structured summary with sections for the claim, opposing positions, a reliability assessment, and a recommended action. This output is designed to be dropped into a review queue, attached to a decision memo, or used as a pre-read before a human reviewer makes a call. Do not use this prompt as the first stage of a pipeline; it will not identify conflicts from raw documents, and feeding it undifferentiated retrieval results will produce vague or misleading summaries.

Avoid this prompt when the dispute is already resolved, when only one source position exists, or when the conflict is trivial and does not warrant human review. If you need to rank evidence by strength before summarizing, use the Evidence Ranking and Prioritization playbook first. If you need to decide whether the conflict matters enough to escalate, use the Conflict Severity Rating Prompt for Evidence Pairs. If the output will go directly to an end user without human review, use the Conflicting Evidence Presentation Prompt for User-Facing Answers instead—this prompt assumes a reviewer is in the loop. Finally, if the domain is high-stakes (clinical, legal, financial audit), ensure the summary includes explicit uncertainty language and a clear statement that the output is a review aid, not a final determination.

This template produces a concise, structured summary of an evidence dispute suitable for operations teams and review queues. It takes a contested claim, opposing source positions, and reliability metadata as inputs, then outputs a summary that includes the nature of the dispute, source-by-source positions, a reliability assessment, and a recommended next step. The output is designed to be actionable for human reviewers who need to triage disputes quickly without reading every source document.

textCopy
You are an evidence dispute analyst. Your task is to produce a structured summary of a disagreement between sources on a specific claim.

## INPUT

**Contested Claim:** [CLAIM]

**Source Positions:**
[SOURCE_POSITIONS]
<!-- Format: For each source, provide:
- Source ID
- Source type (e.g., research paper, news report, regulatory filing, internal document)
- Position on the claim (supports, refutes, partially supports, or neutral)
- Key evidence or quote from the source
- Publication date or recency indicator -->

**Source Reliability Metadata:**
[RELIABILITY_METADATA]
<!-- Format: For each source, provide:
- Authority score or tier
- Recency score
- Methodological quality notes (if applicable)
- Known biases or limitations -->

## OUTPUT SCHEMA

Return a JSON object with the following structure:

{
  "dispute_id": "string - unique identifier for this dispute",
  "contested_claim": "string - the claim under dispute, restated precisely",
  "dispute_type": "string - one of: factual_contradiction, methodological_difference, interpretation_divergence, scope_mismatch, temporal_change, source_error",
  "severity": "string - one of: critical, high, medium, low",
  "positions": [
    {
      "source_id": "string",
      "position": "string - supports | refutes | partially_supports | neutral",
      "key_evidence": "string - the most relevant quote or paraphrase",
      "reliability_summary": "string - one-sentence reliability assessment"
    }
  ],
  "reliability_assessment": {
    "most_reliable_source": "string - source_id",
    "rationale": "string - why this source is most reliable in this context",
    "unresolvable_factors": ["string - factors preventing definitive resolution"]
  },
  "recommended_next_step": "string - one of: escalate_to_domain_expert, request_additional_sources, accept_stronger_source, present_both_positions, defer_decision, mark_as_unresolvable",
  "summary_for_reviewer": "string - 2-3 sentence plain-language summary of the dispute and recommendation",
  "uncertainty_flags": ["string - any caveats about the assessment itself"]
}

## CONSTRAINTS

1. Do not resolve the dispute by picking a winner unless one source is clearly more reliable and the other is demonstrably flawed.
2. When sources are comparably reliable but disagree, recommend presenting both positions.
3. If the dispute type is scope_mismatch or methodological_difference, explain this in the summary_for_reviewer rather than treating it as a factual contradiction.
4. Never fabricate evidence or reliability assessments. If metadata is missing, note it in uncertainty_flags.
5. Keep key_evidence entries verbatim from the source where possible. Use quotes.
6. The summary_for_reviewer must be self-contained and actionable without reading the full input.

## RISK LEVEL

[RISK_LEVEL]
<!-- Use: low, medium, high, critical -->
<!-- High or critical: require human review before any downstream action -->

Adaptation guidance: Replace each square-bracket placeholder with live data from your evidence pipeline. The [SOURCE_POSITIONS] placeholder expects structured source data—wire this to your retrieval or evidence store output. The [RELIABILITY_METADATA] placeholder should pull from your source authority scoring system. If you don't have reliability scores, either omit the reliability_assessment section or flag the gap in uncertainty_flags. The [RISK_LEVEL] placeholder should be set by your application based on the domain and potential impact of acting on disputed evidence. For regulated domains, always set this to high or critical to force human review. Before deploying, validate that the output JSON matches the schema, especially the dispute_type and recommended_next_step enum values. Run eval checks on whether summary_for_reviewer is genuinely actionable and whether key_evidence entries are faithful to the source text.

The Evidence Dispute Summary prompt is designed to sit inside a human review queue, not to resolve disputes autonomously. The output is a structured brief that a reviewer reads before deciding how to handle the conflict. This means the harness must enforce output shape, log every summary for audit, and route the result to the right person or team. The prompt itself does not make decisions—it compresses the dispute into an actionable format.

Integration pattern. Call the prompt after an upstream conflict detection step has identified a contested claim and retrieved the opposing source passages. The typical call flow: (1) a retrieval or RAG pipeline surfaces a claim with conflicting evidence; (2) a conflict detection prompt or rule flags the dispute; (3) this Evidence Dispute Summary prompt receives the claim, the conflicting sources, and any reliability metadata; (4) the output is validated, logged, and pushed into a review queue. Do not call this prompt on every user query—only when a conflict has already been detected. Wrapping the call in a retry with exponential backoff (max 3 attempts) handles transient model failures, but if the output fails schema validation twice, escalate to a human with the raw source passages and a validation_failed flag.

Validation and schema enforcement. The prompt expects a structured output with fields like contested_claim, source_positions (each with source_id, position, key_evidence), reliability_assessment, and recommended_next_step. Use a JSON schema validator in the harness to confirm every required field is present, source_positions is a non-empty array, and recommended_next_step is one of an allowed enum (escalate, defer, request_more_evidence, flag_for_review). If validation passes, attach a unique dispute_id, a timestamp, and the model version to the record before writing to the review queue. If validation fails, log the raw output and the validation errors, then either retry with a repair prompt or escalate. Never surface an unvalidated summary to a reviewer.

Model choice and latency. This is a summarization and structuring task, not a reasoning-heavy one. A capable mid-tier model (GPT-4o, Claude 3.5 Sonnet, or equivalent) is usually sufficient. Latency is not critical here because the output goes to a review queue, not a real-time user response. Set a generous timeout (30–60 seconds) and prefer accuracy over speed. If you are processing high volumes, batch dispute summaries and run them asynchronously. Do not use a small or quantized model for this task—the risk of dropping a source position or misrepresenting a reliability assessment is too high.

Logging and audit trail. Every dispute summary must be logged immutably with: the prompt version, the model and its configuration, the full input (claim + sources), the validated output, the reviewer's eventual decision, and a timestamp chain. This is essential for governance and for debugging when a reviewer disagrees with the summary. If your system allows reviewers to edit the summary before acting, store both the original model output and the edited version. This audit trail is what makes the workflow defensible in regulated or high-stakes environments.

Human-in-the-loop design. The recommended_next_step field is advisory, not binding. The reviewer always has final authority. Design the review UI to show the dispute summary alongside the raw source passages so the reviewer can spot-check the model's compression. Include buttons for common actions (escalate, dismiss, request more evidence) and a free-text field for the reviewer's rationale. Track reviewer override rates by dispute type—if reviewers consistently override the model's recommendation for a particular category, that is a signal to update the prompt or the upstream conflict detection logic.