Multi-Document Quote Aggregation Prompt

This prompt is designed for the moment after retrieval, when your system has pulled back a set of candidate documents and needs to extract, deduplicate, and rank the most relevant verbatim quotes that support a single claim. The primary user is an engineering lead or AI builder integrating a citation-aware RAG pipeline into a product—think legal-tech platforms, enterprise knowledge-base search, or compliance review tools. The job-to-be-done is turning a noisy, multi-source retrieval set into a clean, ranked evidence bundle that downstream answer generation or a human reviewer can trust. You should use this prompt when you have a specific claim to ground, multiple source documents in context, and a need for transparent, auditable provenance tracking.

This prompt is not a general-purpose search or summarization tool. Do not use it when you lack a concrete claim to verify, when your retrieval set is empty or trivially small (fewer than two documents), or when the user expects a synthesized narrative answer rather than a structured evidence set. It is also a poor fit for real-time chat where latency budgets are under one second, because the aggregation, deduplication, and ranking steps require careful model reasoning over potentially large contexts. If your application needs a final answer with inline citations, pair this prompt's output with a downstream RAG Citation Answer Prompt. If you are in a regulated domain such as healthcare or finance, you must route the aggregated quotes through a human review step before they appear in a user-facing response.

Before wiring this into production, define your evaluation criteria clearly: coverage (did we miss a key quote?), redundancy reduction (did we collapse near-duplicates correctly?), and provenance accuracy (is every quote traceable to its source document and span?). The prompt template that follows expects you to supply the claim, the retrieved documents with identifiers, and an output schema. Start by testing it against a golden dataset of claims with known supporting quotes, and measure precision and recall before tuning the deduplication threshold or ranking instructions.

This template is designed to be dropped directly into your application code. It expects a claim to support, a list of documents with their content and metadata, and a set of constraints that define how quotes should be selected, deduplicated, and ranked. The square-bracket placeholders represent variables your application must populate at runtime before sending the request to the model.

textCopy
You are an evidence aggregation specialist. Your task is to extract, deduplicate, and rank supporting quotes from multiple source documents for a single claim.

## CLAIM
[CLAIM]

## SOURCE DOCUMENTS
[DOCUMENTS]
<!--
Each document must include:
- doc_id: unique identifier
- title: document title
- date: publication or revision date
- author: author or source name
- content: full document text or relevant passage
-->

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "claim": "[CLAIM]",
  "evidence_set": [
    {
      "quote_text": "exact verbatim text from source",
      "doc_id": "source document identifier",
      "span_start": character_offset,
      "span_end": character_offset,
      "relevance_score": 0.0_to_1.0,
      "support_strength": "directly_supports | partially_supports | provides_context",
      "deduplication_note": "original | near_duplicate_of_quote_id_X | merged_from_quotes_Y_and_Z"
    }
  ],
  "coverage_assessment": {
    "claim_aspects_covered": ["aspect1", "aspect2"],
    "missing_aspects": ["aspect3"],
    "sufficiency_verdict": "sufficient | partial | insufficient"
  },
  "deduplication_summary": {
    "total_candidate_quotes_found": integer,
    "quotes_after_deduplication": integer,
    "near_duplicate_clusters_merged": integer
  }
}

## CONSTRAINTS
[CONSTRAINTS]
<!--
Example constraints:
- Maximum 10 quotes in final evidence set
- Minimum relevance_score of 0.6 for inclusion
- Prefer quotes from documents published after [DATE_THRESHOLD]
- Prioritize quotes from sources with authority_level >= [AUTHORITY_THRESHOLD]
- Merge near-duplicate quotes when similarity > 0.85
- Flag but exclude contradictory quotes from final set
- Include at least one quote per document when multiple documents are relevant
-->

## DEDUPLICATION RULES
1. Identify quotes that convey the same factual information, even if wording differs.
2. When near-duplicates are found, keep the most specific and well-phrased version.
3. In the deduplication_note field, reference the doc_id of the kept quote for each removed duplicate.
4. If two quotes from different sources provide complementary details on the same point, merge them into a single entry with deduplication_note "merged_from_quotes_X_and_Y".

## RANKING RULES
1. Rank by support_strength first: directly_supports > partially_supports > provides_context.
2. Within the same support_strength tier, rank by relevance_score descending.
3. Break ties by preferring more recent documents, then by source authority.

## INSTRUCTIONS
- Extract only verbatim text. Do not paraphrase or summarize.
- Include character-level span offsets for every quote.
- If no quotes meet the minimum relevance threshold, return an empty evidence_set with sufficiency_verdict "insufficient".
- Do not fabricate quotes. If a document contains no relevant text, do not extract from it.

Before wiring this into production, replace each bracketed placeholder with real data from your retrieval pipeline. The [DOCUMENTS] placeholder is the most critical: your application must format each document with the required fields before insertion. If your retrieval system does not provide character offsets, modify the output schema to use paragraph or section references instead, but be aware that span-level provenance is essential for audit trails and inline citation rendering. For high-stakes domains such as legal or healthcare, add a [HUMAN_REVIEW_FLAG] field to the output schema and route any evidence_set with sufficiency_verdict "partial" or "insufficient" to a review queue before the evidence reaches end users.

This prompt is designed to be called after retrieval, not as a standalone tool. The application layer must first gather candidate quotes from multiple documents using a retrieval pipeline (vector search, keyword search, or hybrid). The prompt then acts as a re-ranking and deduplication engine, consuming a batch of candidate quotes and producing a cleaned, ranked evidence set. The harness is responsible for assembling the input payload, enforcing the output schema, and deciding what to do when the model returns incomplete or low-confidence results.

The input payload should include the target claim in [CLAIM], an array of candidate objects in [CANDIDATE_QUOTES] (each with quote_text, source_document_id, source_span_start, source_span_end, and optional retrieval_score), and a [DEDUPLICATION_THRESHOLD] for near-duplicate detection. The output schema must be enforced at the application level: expect a JSON object with a ranked_quotes array, where each entry contains quote_text, source_document_id, source_span, relevance_score (0-1), is_near_duplicate_of (nullable, referencing another quote ID), and deduplication_rationale. Validate that every returned quote maps back to a provided candidate, that relevance scores are within range, and that deduplication references are self-consistent. If the model returns quotes not present in the input, discard them and log a grounding failure.

For production wiring, implement a retry loop with a maximum of two attempts. On the first failure (malformed JSON, missing fields, or invalid scores), send the raw output and validation errors back to the model in a repair prompt. If the repair also fails, fall back to a deterministic deduplication method (e.g., cosine similarity on embeddings) and return the unranked deduplicated set with a warning flag. Log every aggregation call with the claim hash, input quote count, output quote count, deduplication rate, and any repair attempts. For high-stakes domains like legal or compliance, route outputs with low average relevance scores (below 0.5) or high deduplication rates (above 50%) to a human review queue before the evidence set is used downstream.

Model choice matters here. Use a model with strong instruction-following and JSON output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller models for this task because near-duplicate detection across document boundaries requires semantic comparison that degrades quickly with weaker models. If latency is critical, consider batching: split large candidate sets (over 20 quotes) into overlapping windows of 10-15 quotes, aggregate independently, then run a final deduplication pass across windows. Always include the source_document_id in every output record—this is non-negotiable for audit trails and downstream citation rendering.