Multi-Source Evidence Aggregation Prompt Template

This prompt is designed for verification engineers who need to merge evidence retrieved from multiple sources—such as vector databases, keyword search APIs, and internal knowledge bases—before matching that evidence against claims. The core job-to-be-done is resolving evidence duplication, conflicting provenance, and inconsistent ranking that occurs when independent retrievers return overlapping results for the same claim. The ideal user is a platform engineer or pipeline builder operating a batch verification workflow, not an end-user interacting with a chatbot. You should use this prompt when you have raw evidence sets from two or more retrieval systems and need a single, deduplicated, ranked evidence collection with preserved source attribution. It belongs strictly between the retrieval and claim-evidence matching stages in a verification pipeline.

Do not use this prompt when you have a single retrieval source, as deduplication and provenance merging are unnecessary overhead. It is also inappropriate for real-time, single-query chat scenarios where latency constraints preclude batch aggregation. This prompt assumes that upstream retrievers have already returned structured evidence objects with fields like source_id, text, relevance_score, and retrieval_method. If your retrievers return unstructured text or lack provenance metadata, you must normalize the inputs before invoking this aggregation step. The prompt is not a substitute for a vector database's built-in fusion algorithms; it is a programmatic merge layer for heterogeneous sources that cannot be combined at the retrieval infrastructure level.

Before integrating this prompt into your pipeline, ensure you have defined a clear evidence schema that all upstream retrievers conform to. The output of this prompt feeds directly into claim-evidence matching, so any loss of source attribution or incorrect deduplication will propagate errors downstream. Start by testing with small, known-overlap evidence sets where you can manually verify deduplication accuracy and ranking consistency. Avoid using this prompt as a black box; pair it with eval checks for deduplication precision, source attribution preservation, and ranking correlation with human judgments before scaling to production batch sizes.

This prompt template is the core instruction set for a multi-source evidence aggregation step in your verification pipeline. It is designed to receive evidence chunks from multiple upstream retrievers (e.g., a vector database, a keyword search engine, and a web search API), deduplicate them, rank them by relevance and authority, and produce a clean, unified evidence set with full provenance tracking. The output is intended to be consumed by a downstream claim-to-evidence matching prompt, so the structure must be predictable and machine-readable.

textCopy
You are an evidence aggregation engine in a fact-checking pipeline. Your task is to merge multiple evidence sets, deduplicate semantically equivalent information, rank the remaining evidence by relevance and source authority, and output a structured, deduplicated evidence set with full provenance.

## INPUT
You will receive the following:
- [CLAIM]: A single, atomic claim to verify.
- [EVIDENCE_SOURCES]: A list of evidence sets from different retrievers. Each set has a `source_retriever` name and a list of `chunks`. Each chunk has an `id`, `text`, `source_document_uri`, and `retrieval_score`.

## TASK
1. **Semantic Deduplication:** Identify chunks that convey the same factual information. For each group of near-duplicates, select the single best representative chunk based on the highest `retrieval_score` and most authoritative `source_document_uri`. Do not merge text from different chunks.
2. **Contradiction Flagging:** If two chunks directly contradict each other on a factual point relevant to the claim, flag both for human review instead of selecting one.
3. **Relevance Ranking:** Rank all unique, non-contradicted chunks by their relevance to the [CLAIM]. Use a relevance score from 0.0 to 1.0.
4. **Provenance Tracking:** For every chunk in the final output, preserve its original `id`, `source_document_uri`, and a list of `source_retrievers` that returned it.

## OUTPUT SCHEMA
You must output a single JSON object with the following structure:
{
  "claim": "[The original claim text]",
  "deduplicated_evidence": [
    {
      "chunk_id": "string",
      "text": "string",
      "source_document_uri": "string",
      "relevance_score": 0.0-1.0,
      "source_retrievers": ["string"],
      "is_contradiction": false,
      "contradicts_chunk_id": null
    }
  ],
  "contradiction_flags": [
    {
      "chunk_id_a": "string",
      "chunk_id_b": "string",
      "reason": "Brief explanation of the contradiction."
    }
  ],
  "dedup_summary": {
    "original_chunk_count": 0,
    "deduplicated_chunk_count": 0,
    "contradiction_count": 0
  }
}

## CONSTRAINTS
- Do not fabricate or summarize evidence. Only use text verbatim from the provided chunks.
- If no evidence is provided or all evidence is irrelevant, return an empty `deduplicated_evidence` array and explain in a `processing_notes` field.
- Preserve all original `chunk_id` values exactly as provided.
- If the [RISK_LEVEL] is 'high', you must flag any evidence from sources not in the [AUTHORIZED_SOURCE_LIST] for human review.

To adapt this template, replace the square-bracket placeholders with your pipeline's runtime data. The [CLAIM] should be a single, atomic string from your claim extraction step. The [EVIDENCE_SOURCES] payload must be a pre-serialized JSON array matching the described structure. For high-stakes domains like healthcare or finance, set [RISK_LEVEL] to 'high' and provide an [AUTHORIZED_SOURCE_LIST] to enforce source allowlisting. After pasting this prompt into your verification harness, run it against a golden dataset of known duplicates and contradictions to calibrate the deduplication accuracy before production deployment.

The multi-source evidence aggregation prompt is not a standalone tool; it is a processing stage inside a larger verification pipeline. In production, this prompt sits between evidence retrieval and claim-to-evidence matching. It receives raw evidence sets from multiple retrievers—vector search, keyword search, API calls, knowledge base queries—and must produce a deduplicated, ranked, and provenance-tracked evidence bundle before matching begins. The implementation harness must enforce input contracts (each evidence item requires source_id, retrieval_method, content, and retrieval_timestamp), validate output shape (deduplicated list with canonical_id, merged_sources, rank_score, and provenance array), and handle failures without corrupting downstream stages.

Wire this prompt into your pipeline with a pre-processing validator, a retry wrapper, and a post-processing schema check. The pre-validator should reject inputs missing required fields or containing evidence from unauthorized sources. Use a retry policy with exponential backoff (initial delay 1s, max 3 retries) for transient model failures, but fail fast on schema violations—retrying a malformed input will not help. After the model responds, run a strict output validator that checks: every canonical_id is unique, every merged_sources entry references a valid input source_id, no evidence content is fabricated (compare hashes of input content against output content), and rank_score values fall within the expected range. For high-stakes verification domains, route outputs where deduplication confidence is below a threshold (e.g., rank_score < 0.7) to a human review queue with the original evidence sets attached. Log every aggregation run with pipeline_run_id, claim_id, input_source_count, output_deduplicated_count, dedup_ratio, latency_ms, and any validation failures for observability dashboards.

Choose your model tier based on evidence volume and complexity. For batches under 50 evidence items with straightforward text, a fast model like GPT-4o-mini or Claude Haiku works well. For larger sets, cross-lingual evidence, or high-precision requirements, use a more capable model and consider splitting the batch into chunks with overlapping windows to catch cross-boundary duplicates. Avoid the temptation to skip provenance tracking—every merged evidence item must carry forward all original source_id values so downstream matching can cite specific sources. If your pipeline includes a human review stage, pass the full provenance array to reviewers so they can assess source quality directly. The next step after aggregation is claim-to-evidence matching; ensure your output schema aligns with the matcher's expected input format to avoid contract breaks at the stage boundary.