Evidence Reconciliation Prompt for Analyst Reports

This prompt is designed for research analysts and AI systems that produce reconciled reports from multiple conflicting sources. It attempts to explain apparent contradictions by identifying scope differences, methodological variations, temporal factors, or population differences rather than forcing a false consensus. The core job-to-be-done is structured conflict resolution: given two or more evidence sources that appear to contradict each other, produce an analysis that determines whether the conflict is genuine or explainable through non-substantive differences in how the evidence was generated or framed.

Use this prompt only after a conflict detection step has already identified the disagreement. It belongs in a pipeline downstream of a contradiction extraction or conflict detection prompt—never as a first-pass synthesis tool. The ideal user is building a research assistant, decision-support system, or evidence audit workflow where unresolved conflicts would produce misleading or overconfident answers. Required context includes the specific conflicting claims, the full text of each source passage, source metadata (date, methodology, population, scope), and the question or claim being evaluated. Without source metadata, the model cannot reliably distinguish methodological differences from genuine factual disputes.

Do not use this prompt when sources are already aligned, when you need a general synthesis or summarization, or when the disagreement is clearly a factual error in one source rather than an explainable divergence. This prompt is also inappropriate for real-time conversational settings where latency constraints prevent thorough analysis. If the conflict involves regulated domains such as healthcare, finance, or legal evidence, the output must pass through human review before being surfaced to users. The prompt's failure mode is forcing reconciliation when genuine disagreement exists—always pair it with an eval that checks whether the reconciliation explanation is grounded in specific source details rather than speculative harmonization.

This prompt template is designed for research analysts who need to produce reconciled reports from conflicting evidence sources. It attempts to identify scope differences, methodological variations, temporal factors, or population differences that explain apparent contradictions. The template is structured to force explicit reasoning about whether reconciliation is possible or whether genuine disagreement must be surfaced to the reader. Replace each square-bracket placeholder with your specific data before sending to the model.

codeCopy
You are a research analyst producing a reconciled evidence report. Your task is to examine conflicting evidence on a specific claim or question and determine whether the apparent contradictions can be resolved through careful analysis of scope, methodology, timing, or population differences.

## INPUT

**Claim or Question Under Review:**
[CLAIM_OR_QUESTION]

**Evidence Sources:**
[EVIDENCE_SOURCES]

**Domain Context:**
[DOMAIN_CONTEXT]

## CONSTRAINTS

[CONSTRAINTS]

## OUTPUT SCHEMA

Produce a JSON object with the following structure:

```json
{
  "reconciliation_status": "reconciled | partially_reconciled | unresolved_conflict | insufficient_evidence",
  "reconciled_synthesis": "string or null if unresolved",
  "unresolved_disagreements": [
    {
      "disputed_point": "string",
      "position_a": {
        "claim": "string",
        "sources": ["source_id"],
        "supporting_evidence_quality": "high | medium | low"
      },
      "position_b": {
        "claim": "string",
        "sources": ["source_id"],
        "supporting_evidence_quality": "high | medium | low"
      },
      "reconciliation_attempt": "string explaining why reconciliation failed",
      "recommended_presentation": "present_both_sides | defer_to_stronger_evidence | flag_for_human_review"
    }
  ],
  "reconciliation_factors_identified": [
    {
      "factor_type": "scope_difference | methodological_difference | temporal_factor | population_difference | definitional_variance | other",
      "description": "string",
      "sources_involved": ["source_id"],
      "confidence": "high | medium | low"
    }
  ],
  "evidence_quality_assessment": {
    "overall_quality": "high | medium | low | mixed",
    "strongest_sources": ["source_id"],
    "weakest_sources": ["source_id"],
    "quality_notes": "string"
  },
  "limitations_and_caveats": ["string"],
  "recommended_next_steps": ["string"]
}

INSTRUCTIONS

1. First, identify all points of apparent conflict across the evidence sources.
2. For each conflict, attempt reconciliation by examining:
   • Scope differences: Do sources address different populations, time periods, or conditions?
   • Methodological differences: Do different study designs, measurement approaches, or analytical methods explain divergence?
   • Temporal factors: Could timing of data collection or publication explain the difference?
   • Population differences: Do sources study different groups, segments, or contexts?
   • Definitional variance: Do sources define key terms or metrics differently?
3. If reconciliation is possible, produce a synthesized statement that accounts for the differences.
4. If reconciliation is not possible, clearly identify the unresolved disagreement and recommend how to present it.
5. Do not force reconciliation when genuine disagreement exists. It is better to surface unresolved conflict than to manufacture false consensus.
6. Assess the quality of each source and use quality differentials to weight evidence appropriately.
7. Flag any evidence gaps that prevent confident reconciliation.

EXAMPLES

[EXAMPLES]

RISK LEVEL

[RISK_LEVEL]

Adaptation guidance: The [EVIDENCE_SOURCES] placeholder should contain structured source objects with at minimum an id, content, publication_date, and methodology field. The [CONSTRAINTS] placeholder is where you inject domain-specific rules, such as regulatory requirements, organizational policies, or output formatting restrictions. The [EXAMPLES] placeholder is critical for this prompt—provide at least one example of successful reconciliation and one example of correctly identifying an unresolved conflict. The [RISK_LEVEL] field should be set to high when the output influences consequential decisions, which triggers additional validation requirements in the implementation harness. After generating output, validate that reconciliation_status matches the content: if unresolved_conflict, the unresolved_disagreements array must be non-empty; if reconciled, the reconciled_synthesis field must be populated.

What to do next: Copy this template into your prompt management system and replace each placeholder with your actual data. Before deploying to production, run this prompt against a golden dataset of known conflicts where you have ground-truth labels for whether reconciliation is possible. Measure both false reconciliation (forcing agreement where genuine disagreement exists) and false conflict (failing to identify reconcilable differences). If your domain involves regulated decisions, always route unresolved_conflict outputs to human review before publication.

The Evidence Reconciliation Prompt is designed to be called after evidence conflict detection has already identified contradictory passages. In a production pipeline, this prompt sits downstream of a conflict detection step and upstream of a report generation or decision-support step. The typical call pattern is: retrieve evidence → detect conflicts → for each conflict cluster, call the reconciliation prompt → aggregate reconciled findings into the final analyst report. Because reconciliation can force false consensus when genuine disagreement exists, the harness must include a reconciliation refusal detector that flags outputs where the model over-harmonizes contradictory evidence.

Validation and retry logic should focus on two failure modes. First, validate that every reconciliation claim references at least two specific sources and identifies the reconciling factor (scope, methodology, temporal, or population difference). Use a structured output schema with required fields: [CLAIM_A], [CLAIM_B], [RECONCILING_FACTOR], [RECONCILIATION_CONFIDENCE], and [UNRESOLVED_FLAG]. If UNRESOLVED_FLAG is true, route the output to a human review queue instead of the synthesis step. Second, implement a semantic contradiction check using an LLM judge that compares the original conflicting claims against the reconciled output. If the judge detects that the reconciliation dismisses or misrepresents one side, increment a retry counter (max 2 retries) with a revised prompt that includes the judge's feedback as [PRIOR_FAILURE_FEEDBACK]. Log all reconciliation attempts, judge scores, and retry decisions for auditability.

Model choice and tool integration matter here. Use a model with strong reasoning capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent) because reconciliation requires nuanced comparison of claims. For high-stakes domains like financial or clinical analyst reports, pair the prompt with a retrieval-augmented generation (RAG) step that fetches the full source documents for each conflicting passage before reconciliation, ensuring the model has complete context rather than just extracted snippets. If your application uses a multi-agent architecture, consider a dedicated Reconciliation Agent that owns this prompt, receives conflict clusters from a Detection Agent, and passes reconciled or flagged-unresolved outputs to a Synthesis Agent. The Reconciliation Agent should have access to a search_source_document tool for deep-dive verification. Always include a human approval gate when RECONCILIATION_CONFIDENCE falls below a threshold (e.g., <0.7) or when UNRESOLVED_FLAG is true. The approval interface should display the original conflicting claims, the proposed reconciliation, and the reconciling factor, allowing an analyst to accept, reject, or modify before the output enters the final report.