Source Conflict Resolution and Ranking Prompt

This prompt is designed for the evidence selection stage of a Retrieval-Augmented Generation (RAG) pipeline, specifically when your knowledge base contains documents that disagree on material facts, dates, figures, or interpretations. The core job-to-be-done is to produce a structured, auditable resolution of contradictory evidence before it reaches the answer synthesis step. The ideal user is an AI engineer or technical decision maker building a RAG system for legal, financial, or regulatory domains where surfacing conflict is a feature, not a bug, and where hiding disagreement creates unacceptable compliance or decision risk. You should use this prompt when your retrieval step returns multiple passages that may contradict each other, and you need a deterministic, traceable method to rank them by agreement, source authority, and resolution status.

The prompt assumes you already have a set of retrieved passages and their associated metadata, including source identifiers, publication dates, and authority classifications. It does not perform retrieval itself, nor does it generate a final answer. Instead, it acts as a conflict-aware re-ranker that applies majority-agreement scoring, source authority weighting, and explicit flagging of unresolved conflicts. The output is a ranked list with scores and conflict flags, which your downstream synthesis prompt can use to decide what to cite, what to caveat, and what to present as contested. This separation of concerns is critical: conflict resolution is a distinct reasoning step that should be logged and audited independently of the final answer generation.

Do not use this prompt when your knowledge base is homogeneous, curated, or known to be conflict-free. In those cases, a simpler relevance or credibility ranking prompt will be more cost-effective and faster. Also avoid this prompt for real-time chat applications where latency is critical and the conflict resolution step adds unacceptable delay. If your system cannot tolerate surfacing any disagreement to the end user, this prompt is not the right tool; you would instead need a prompt that selects a single 'best' answer and suppresses alternatives. Finally, if your source documents lack authority metadata, the source authority weighting component will be ineffective, and you should either enrich your metadata pipeline or use a simpler conflict-flagging prompt that relies solely on content analysis.

This prompt template is designed for legal, financial, and compliance RAG systems where retrieved passages may contradict each other. It instructs the model to identify conflicts, weight sources by authority and recency, compute majority-agreement scores, and produce a conflict-resolved ranking. The output includes explicit flags for unresolved conflicts so downstream answer synthesis can decide whether to present competing views, request more evidence, or escalate for human review. Replace each square-bracket placeholder with your actual data before sending the prompt to the model.

textCopy
You are a source conflict resolution engine operating inside a regulated RAG system. Your output will be audited.

## INPUT
Query: [USER_QUERY]

Retrieved Passages:
[PASSAGES_ARRAY]

## SOURCE AUTHORITY WEIGHTS
[SOURCE_AUTHORITY_MAP]

## OUTPUT SCHEMA
Return valid JSON matching this schema exactly:
{
  "conflict_resolution": {
    "total_passages": <int>,
    "conflicts_detected": <int>,
    "unresolved_conflicts": <int>,
    "resolution_summary": "<string describing overall conflict picture>"
  },
  "ranked_passages": [
    {
      "passage_id": "<string>",
      "rank": <int starting at 1>,
      "majority_agreement_score": <float 0.0 to 1.0>,
      "authority_weight": <float 0.0 to 1.0>,
      "recency_weight": <float 0.0 to 1.0>,
      "composite_score": <float 0.0 to 1.0>,
      "conflicting_passage_ids": ["<string>", ...],
      "conflict_severity": "none" | "minor" | "major" | "direct_contradiction",
      "resolution_status": "resolved" | "unresolved" | "majority_resolved" | "authority_override",
      "resolution_rationale": "<string explaining how conflict was resolved or why it remains unresolved>",
      "key_claim": "<string: the core factual claim from this passage relevant to the query>"
    }
  ],
  "unresolved_conflict_report": [
    {
      "conflict_id": "<string>",
      "passage_ids_in_conflict": ["<string>", ...],
      "conflict_description": "<string describing the contradictory claims>",
      "recommended_action": "escalate_to_human" | "present_both_views" | "retrieve_more_evidence" | "flag_as_uncertain",
      "risk_level": "low" | "medium" | "high" | "critical"
    }
  ],
  "audit_trail": {
    "conflict_detection_method": "<string>",
    "weighting_rationale": "<string>",
    "exclusion_decisions": [
      {
        "passage_id": "<string>",
        "reason": "<string>"
      }
    ]
  }
}

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## INSTRUCTIONS
1. Read all passages and identify factual claims relevant to the query.
2. Detect contradictions between passages. A contradiction exists when two passages assert mutually exclusive facts about the same subject.
3. For each conflict, attempt resolution using these rules in order:
   a. Majority agreement: if 3+ independent sources agree, the majority view wins.
   b. Authority override: if a source with authority_weight >= 0.8 conflicts with lower-authority sources, the high-authority source wins.
   c. Recency tiebreaker: if authority is comparable, prefer the more recent source.
   d. If none of the above resolve the conflict, mark it as unresolved.
4. Compute composite_score as: (majority_agreement_score * 0.4) + (authority_weight * 0.35) + (recency_weight * 0.25).
5. Rank passages by composite_score descending.
6. Never fabricate resolution. If evidence is insufficient to resolve a conflict, mark it unresolved.
7. Include every input passage in ranked_passages, even if ranked low.
8. If [RISK_LEVEL] is "high" or "critical", flag all unresolved conflicts for human review.

Adaptation notes: Replace [PASSAGES_ARRAY] with a JSON array of passage objects, each containing at minimum passage_id, text, source_id, and publication_date. [SOURCE_AUTHORITY_MAP] should map each source_id to an authority weight between 0.0 and 1.0, pre-computed by your credibility assessment pipeline. [CONSTRAINTS] should include domain-specific rules such as regulatory precedence, jurisdiction hierarchy, or temporal cutoff dates. [FEW_SHOT_EXAMPLES] should contain 2-3 worked examples showing correct conflict resolution and ranking for your domain. [RISK_LEVEL] should be set dynamically based on the query context—use "high" or "critical" for financial reporting, legal filings, or clinical decisions. Before deploying, validate the output JSON against the schema and run at least 20 test cases with known conflicts to calibrate your authority weights and composite score thresholds.

The Source Conflict Resolution and Ranking Prompt is designed to sit between retrieval and answer synthesis in high-stakes RAG pipelines—legal document review, financial compliance checks, and regulatory filings where contradictory evidence cannot be silently discarded. The prompt expects a pre-retrieved set of passages that may conflict, along with optional source metadata (authority level, publication date, document type). The output is a structured conflict-resolved ranking with majority-agreement scoring, source authority weighting, and explicit flags for unresolved conflicts. This harness must enforce that the model never fabricates a resolution when evidence is genuinely contradictory; it must surface the conflict instead.

Wire this prompt as a pre-synthesis gate in your RAG pipeline. After retrieval returns candidate passages, pass them through a deduplication step first (see the Multi-Passage Deduplication and Selection Prompt), then into this conflict resolution prompt. The model receives the deduplicated passage set, source metadata, and the original user query. Parse the JSON output into a ConflictResolution schema with fields: resolved_ranking (ordered list of passages with conflict-adjusted scores), majority_agreement_score (float 0–1), unresolved_conflicts (array of conflict objects with passage pairs and the disputed claim), and resolution_audit_trail (step-by-step reasoning). Validate the output against this schema immediately. If validation fails, retry once with the error message injected into the prompt as a correction hint. If the retry also fails, route to a human review queue with the raw passages and partial output attached.

Audit trail generation is mandatory for regulated use cases. Store the full prompt input, model output, schema validation result, and any retry attempts in an append-only log with a unique conflict_resolution_id. This ID should propagate downstream into the answer synthesis step and final citation metadata. When the system cites a source, the citation must reference both the source document and the conflict resolution ID that determined its ranking. This creates a traceable chain: retrieval → deduplication → conflict resolution → answer synthesis. For financial audit or legal discovery workflows, this audit trail is what turns an AI-assisted ranking into a defensible process artifact.

Model choice matters here. Use a model with strong reasoning capabilities (GPT-4, Claude 3.5 Sonnet, or equivalent) because conflict resolution requires comparing claims across passages, detecting subtle contradictions, and weighing source authority. Smaller or faster models often collapse contradictory evidence into a false consensus or ignore minority-view sources entirely. If latency constraints force a smaller model, add a second-pass verification step: after the initial ranking, run the Evidence Sufficiency Check Prompt to confirm that unresolved conflicts are still flagged and that no contradictory passage was silently dropped. Set a confidence threshold—if the majority agreement score falls below 0.6 or unresolved conflicts exceed 20% of passages, escalate to human review regardless of schema validity.

Common failure modes to monitor: (1) The model resolves conflicts by inventing a middle-ground claim not present in any source—validate that every claim in the resolution audit trail maps to at least one passage. (2) Source authority weighting overpowers factual correctness—a high-authority but outdated source may contradict a lower-authority but recent source; check that recency is factored into the weighting logic. (3) The model flags too many trivial conflicts (stylistic differences, minor date discrepancies) as unresolved, flooding the review queue—tune the prompt's [CONSTRAINTS] to specify a materiality threshold. Log all three failure patterns and review them weekly to decide whether the prompt needs adjustment or the retrieval step needs better source filtering.