Source Conflict Triage Prompt for Decision-Support Systems

This prompt is for decision-support system builders who need to triage evidence conflicts by urgency and impact before they reach a user-facing answer or a downstream synthesis step. It takes a set of pre-detected conflicts and classifies each into an action category: escalate for immediate human review, defer with a note for later resolution, or dismiss as low-stakes noise. The core job-to-be-done is replacing ad-hoc, inconsistent judgment with a repeatable, auditable triage decision that your application can act on programmatically. Ideal users are engineering leads and AI product teams building research assistants, analyst tools, or compliance review systems where conflicting sources are common and the cost of getting triage wrong is high.

Use this prompt when your pipeline already identifies contradictions but lacks a consistent, auditable way to decide which conflicts matter. You must provide pre-extracted conflict descriptions, source metadata, and domain context. The prompt assumes conflicts have already been detected and described by an upstream process such as a contradiction extraction prompt or a multi-source conflict detection step. It does not resolve conflicts. It only decides what to do about them. Do not use this prompt as a conflict detector. If you feed it raw passages and expect it to find contradictions, you will get unreliable results because the prompt is designed for classification, not discovery. Also avoid using this prompt when the triage decision requires real-time human judgment on nuanced content. The prompt can pre-sort and recommend, but high-stakes domains such as clinical safety or legal filings should always route escalate decisions to a human review queue with full source evidence attached.

Before wiring this into production, define clear escalation criteria for your domain. What constitutes high impact? Is it financial exposure above a threshold, patient safety risk, regulatory non-compliance, or public reputation damage? Encode these criteria in the [CONSTRAINTS] and [RISK_LEVEL] placeholders. Also prepare eval cases: known high-severity conflicts that must trigger escalate, known low-stakes disagreements that should be dismissed, and ambiguous cases where defer is the correct call. Run these through the prompt before deployment and measure precision and recall on each triage category. If your system cannot tolerate a misclassified escalate, add a human-in-the-loop gate after the prompt output. The next section provides the copy-ready template you can adapt.

This template is designed to be dropped directly into your decision-support system's prompt layer. It assumes you have already detected a set of conflicting claims across sources and now need to classify each conflict by severity, determine which require immediate human review, and produce a structured triage report. Replace every square-bracket placeholder with your actual data before sending. The prompt enforces a strict output schema so your application can parse the triage decisions programmatically and route high-severity conflicts to the appropriate escalation queue.

textCopy
You are a source conflict triage specialist for a decision-support system. Your job is to assess detected evidence conflicts and classify each by urgency, impact, and required action.

## INPUT

CONFLICT LIST:
[CONFLICT_LIST]

CONTEXT:
[DECISION_CONTEXT]

RISK TOLERANCE:
[RISK_TOLERANCE]

## CONFLICT LIST SCHEMA
Each conflict in the input is a JSON object with:
- conflict_id: string
- claim: the disputed statement
- source_a: object with source_id, excerpt, reliability_score (0-1), recency (date)
- source_b: object with source_id, excerpt, reliability_score (0-1), recency (date)
- conflict_type: one of "factual_contradiction", "methodological_difference", "interpretation_divergence", "scope_mismatch", "temporal_change"

## TRIAGE RULES

1. Assign a severity score from 1 (low) to 5 (critical) based on:
   - Impact on decision quality if unresolved
   - Reliability gap between conflicting sources
   - Reversibility of the decision this evidence supports
   - Whether the conflict is resolvable with available information

2. Assign an action from: "escalate_immediate", "flag_for_review", "note_and_defer", "ignore_low_stakes"

3. Escalation criteria (action must be "escalate_immediate" if ANY of these are true):
   - Severity >= 4
   - The conflict affects a safety-critical, legal, or financial determination
   - Both sources have reliability_score >= 0.7 but directly contradict each other on a material fact
   - The decision context indicates the decision is irreversible or high-cost

4. For "note_and_defer" conflicts, include a brief deferral rationale and suggested revisit conditions.

5. For "ignore_low_stakes" conflicts, explain why the disagreement does not materially affect the decision.

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "triage_summary": {
    "total_conflicts": number,
    "escalate_immediate_count": number,
    "flag_for_review_count": number,
    "note_and_defer_count": number,
    "ignore_low_stakes_count": number,
    "highest_severity": number
  },
  "conflicts": [
    {
      "conflict_id": "string",
      "severity": number,
      "severity_rationale": "string (2-3 sentences)",
      "action": "escalate_immediate" | "flag_for_review" | "note_and_defer" | "ignore_low_stakes",
      "escalation_target": "string | null (role or team to escalate to, required if action is escalate_immediate)",
      "deferral_rationale": "string | null (required if action is note_and_defer)",
      "revisit_conditions": "string | null (required if action is note_and_defer)",
      "ignore_rationale": "string | null (required if action is ignore_low_stakes)",
      "resolution_notes": "string (any additional context for the reviewer)"
    }
  ]
}

## CONSTRAINTS

- Do not resolve or reconcile the conflicts. Your job is triage only.
- Do not downgrade severity because a conflict seems easy to resolve. Rate based on impact if left unresolved.
- If reliability_score is missing for a source, treat it as 0.5 and note this in resolution_notes.
- If recency is missing, do not assume staleness; note the missing data in resolution_notes.
- Never output markdown fences around the JSON. Return raw JSON only.

To adapt this template, start by mapping your conflict detection output into the CONFLICT_LIST schema. If your upstream detection prompt produces a different structure, add a transformation step in your application layer rather than complicating this triage prompt. The DECISION_CONTEXT placeholder should describe what decision the evidence supports, who the decision-maker is, and the cost of being wrong—this context is what makes severity scoring meaningful. The RISK_TOLERANCE field accepts values like "low", "medium", or "high" and should influence the threshold at which conflicts escalate. If your system has defined escalation targets (specific teams, roles, or queues), you can hardcode them into the escalation criteria section or pass them as an additional [ESCALATION_MAP] variable. Test this prompt with at least three conflict scenarios before production: a clear high-severity contradiction, a low-stakes methodological difference, and an edge case where one source has missing reliability data. Verify that the output JSON parses correctly and that escalation decisions match your organization's actual review workflows.

This prompt is designed to sit downstream of a conflict detection step and upstream of a resolution or escalation workflow. It expects a structured list of detected conflicts, each with source pairs, conflicting claims, and a preliminary conflict type. The output is a triage decision per conflict: resolve now, note and defer, or ignore. The harness must enforce that only conflicts meeting the defined impact and urgency thresholds reach human reviewers or automated resolution paths.

Integration pattern: Place this prompt after your conflict detection module and before any resolution logic. The input should be assembled programmatically from your conflict store, mapping each conflict to the [CONFLICT_LIST] placeholder as a JSON array of objects with fields: conflict_id, claim, source_a, source_b, conflict_type, and domain. The [IMPACT_CRITERIA] placeholder should be populated from your application's risk taxonomy—for example, mapping high_impact to conflicts affecting patient safety, regulatory compliance, or financial thresholds above a defined dollar amount. The [ESCALATION_RULES] placeholder defines who reviews what: route resolve_now items to a review queue with SLA deadlines, log note_and_defer items to a deferred conflict register with a revisit date, and discard or archive ignore items with an audit note. Validate the model's output against a strict JSON schema before acting on any triage decision. A malformed severity rating or missing escalation_target field should trigger a retry with the error message injected into the prompt context, not a silent fallback.

Model choice and latency: Use a model with strong instruction-following and JSON output capabilities for this task. The triage logic requires consistent application of impact criteria across conflicts, so prefer models with low variance on structured classification tasks. If your conflict volume is high, batch conflicts into groups of 10-20 per prompt call to balance latency against consistency—larger batches risk the model losing fidelity on edge cases. Log every triage decision with the model version, prompt hash, input conflict IDs, and output decisions. For high-impact domains, require human confirmation on all resolve_now items before any downstream action is taken. Never auto-resolve a conflict that the model flags as high-severity without a human in the loop.

Testing and failure modes: Before production, run this prompt against a golden set of 50-100 labeled conflicts covering all severity levels and conflict types your system encounters. Measure agreement with human triage decisions, paying special attention to false negatives—conflicts the model marks as ignore or note_and_defer that human reviewers would escalate. The most common failure mode is severity miscalibration under domain shift: a conflict that looks low-impact in a generic context may be high-impact in your specific domain. Mitigate this by keeping [IMPACT_CRITERIA] tightly scoped to your application's risk definitions and updating it when your product surface changes. A secondary failure mode is inconsistent triage across similar conflicts; detect this by comparing triage decisions for conflicts with the same conflict type and similar source authority gaps. If variance exceeds a threshold, refine the prompt's decision rules or add few-shot examples of borderline cases.