Inferensys

Prompt

Multi-Source Citation Merging Prompt

A practical prompt playbook for merging overlapping citations from multiple documents into a single, deduplicated attribution chain with source priority conflict resolution.
Stylish WeWork-like workspace with hot desks and document wall, professional searching through enterprise knowledge base on a mounted ultrawide display, warm industrial pendants overhead.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job, the ideal user, required inputs, and the boundaries where this prompt should not be applied.

This prompt is designed for knowledge base operators and RAG system builders who need to synthesize a single, coherent answer from multiple overlapping source documents. The core job-to-be-done is merging citations from several retrieved chunks into a single, deduplicated attribution chain. The ideal user is an AI engineer or product developer who already has a retrieval pipeline in place and now faces the downstream problem of presenting a unified, auditable answer to the end-user without repeating the same source or burying the reader in redundant footnotes.

Use this prompt when your retrieval step returns multiple documents that cover the same factual territory—for example, three internal policy documents that all define a company holiday schedule, or two technical specs that describe the same API parameter with minor variations. The prompt requires the following inputs: a user question, a set of retrieved documents with unique identifiers, and a defined output schema for citations. It is not suitable for single-document Q&A, for scenarios where source documents are guaranteed to be non-overlapping, or for real-time chat applications where the latency of a merge-and-deduplicate step would violate user experience constraints. If your sources are highly contradictory rather than overlapping, use the Source Conflict Disclosure Prompt instead.

Before wiring this into production, ensure you have a reliable method for assigning unique, stable identifiers to every document chunk. The prompt's deduplication logic depends on these IDs to recognize when two passages are the same source. You should also define a conflict resolution priority—such as 'prefer the most recent document' or 'prefer the document with the highest retrieval score'—and pass that rule into the prompt's [CONSTRAINTS] placeholder. Without explicit priority rules, the model will make arbitrary choices that may not match your business logic. Always pair this prompt with a post-generation validation step that checks for duplicate citations and verifies that every cited claim appears in the referenced source.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Multi-Source Citation Merging Prompt delivers value and where it introduces risk. Use these cards to decide if this prompt fits your pipeline before you integrate it.

01

Good Fit: Overlapping Knowledge Bases

Use when: your retrieval returns multiple documents covering the same fact, and you need a single, deduplicated citation chain. Guardrail: run a pre-check to confirm that at least two retrieved chunks share a semantic overlap before invoking the merge prompt.

02

Bad Fit: Single-Source Answers

Avoid when: only one document is retrieved or the question is answered entirely within a single chunk. Risk: the merge prompt may hallucinate conflicts or fabricate additional sources to satisfy the 'multi-source' expectation. Guardrail: bypass the merge prompt and use a standard citation prompt when the retrieval set contains only one unique source.

03

Required Inputs

What you need: a list of retrieved passages with document IDs, chunk indices, and relevance scores. Guardrail: validate that every input passage has a unique, stable identifier before calling the prompt. Missing IDs will break the deduplication logic and produce untraceable citations.

04

Operational Risk: Source Priority Conflicts

What to watch: when two sources disagree and the model silently picks a winner without disclosing the conflict. Guardrail: require the prompt to output a conflict_flag boolean and a conflict_note string for any contradictory evidence. Log these flags for human review in regulated domains.

05

Operational Risk: Citation ID Drift

What to watch: the model may renumber, truncate, or hallucinate citation IDs during the merge, breaking downstream traceability. Guardrail: post-process the output to verify that every returned citation ID exists in the input set. Reject and retry if any ID is missing or malformed.

06

Latency and Cost Sensitivity

What to watch: merging citations across many sources adds token overhead and inference time, especially with long context windows. Guardrail: cap the number of input passages (e.g., 10–15) and use a lightweight model for the merge step. Escalate to a more capable model only when source conflicts are detected.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt for merging overlapping citations from multiple source documents into a single, deduplicated, and prioritized attribution chain.

The following prompt template is designed to be dropped into a RAG pipeline after retrieval but before final answer synthesis. It accepts a list of source documents and a set of claims or a draft answer, then produces a deduplicated citation map. Use it when your retrieval step returns multiple chunks that reference the same underlying fact, or when different versions of a document create conflicting attribution chains. The template uses square-bracket placeholders for all dynamic inputs, making it safe to copy, adapt, and parameterize in code.

text
You are a citation merging engine. Your job is to take a set of claims and a list of source documents, then produce a deduplicated, prioritized citation map.

## INPUT

### Claims or Draft Answer
[CLAIMS_OR_DRAFT_ANSWER]

### Source Documents
[SOURCE_DOCUMENTS]

Each source document is a JSON object with the following fields:
- id: a unique document identifier
- title: the document title
- date: the publication or revision date in ISO 8601 format
- priority: an integer from 1 (highest) to 5 (lowest) indicating source authority
- text: the relevant passage text

## TASK

1. For each claim in the input, identify all source documents that support it.
2. If multiple sources support the same claim, select the best citation using these rules in order:
   a. Prefer the source with the highest priority (lowest integer).
   b. If priorities are equal, prefer the most recent date.
   c. If dates are equal, prefer the source with the most specific supporting text (longest relevant passage).
3. If two sources contradict each other on a claim, do not merge them. Flag the conflict.
4. Remove duplicate citations where the same source is cited multiple times for the same claim.
5. For each unique claim-to-source mapping, extract the minimal verbatim quote that supports the claim (no more than 50 words).

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "merged_citations": [
    {
      "claim": "The exact claim text from the input",
      "citation": {
        "source_id": "string",
        "source_title": "string",
        "source_date": "ISO 8601 string",
        "quote": "Verbatim supporting text, max 50 words",
        "priority": integer
      },
      "conflict": null or {
        "conflicting_source_id": "string",
        "conflicting_quote": "string",
        "conflict_note": "Brief description of the contradiction"
      }
    }
  ],
  "deduplication_summary": {
    "total_claims": integer,
    "unique_citations": integer,
    "duplicates_removed": integer,
    "conflicts_found": integer
  }
}

## CONSTRAINTS

- Do not invent claims not present in the input.
- Do not cite a source that does not support the claim.
- If no source supports a claim, include it in the output with citation set to null and add a note.
- Keep quotes verbatim. Do not paraphrase inside the quote field.
- If the input contains more than [MAX_CLAIMS] claims, process only the first [MAX_CLAIMS] and note the truncation.

To adapt this template for your system, replace the placeholders as follows: [CLAIMS_OR_DRAFT_ANSWER] should receive the output of your answer generation step, split into individual factual claims (use a separate claim-decomposition prompt if needed). [SOURCE_DOCUMENTS] should be populated with the top-k retrieved chunks from your vector store or search index, formatted as the JSON objects described. [MAX_CLAIMS] is a safety limit—set it based on your latency budget and token limits (start with 50 and tune from there). If your source documents lack a priority field, either add one via metadata or remove that tiebreaker rule from the prompt. Always validate the output JSON against the schema before passing citations to the final answer formatter. For high-stakes domains, route outputs with conflicts to a human reviewer before publishing.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Multi-Source Citation Merging Prompt. Each placeholder must be populated before the prompt is assembled and sent. Validation notes describe how to verify the input is well-formed before execution.

PlaceholderPurposeExampleValidation Notes

[RETRIEVED_DOCUMENTS]

Array of source documents with text, metadata, and retrieval scores that may contain overlapping or redundant citations

[{"doc_id": "doc-42", "text": "...", "score": 0.92, "source": "knowledge-base-v2"}]

Must be a non-empty array. Each object requires doc_id, text, and source fields. score is optional but recommended for priority resolution. Reject if array is empty or any document lacks text.

[CITATION_FORMAT]

Target citation style or output schema for merged citations

"inline-bracketed" or "json-citation-object" or "apa-inline"

Must match one of the allowed enum values: inline-bracketed, json-citation-object, apa-inline, mla-inline, chicago-footnote, custom-schema. Reject unrecognized formats before prompt assembly.

[OUTPUT_SCHEMA]

Expected structure for the merged citation output when using machine-readable formats

{"citations": [{"source_ids": ["doc-42", "doc-17"], "merged_text": "...", "confidence": "high"}]}

Required when [CITATION_FORMAT] is json-citation-object or custom-schema. Must be a valid JSON Schema object. Validate parseability before injection. Null allowed for prose-only formats.

[DEDUPLICATION_STRATEGY]

Rule for handling duplicate or near-duplicate citations across sources

"keep-highest-score" or "keep-earliest-date" or "merge-all"

Must be one of: keep-highest-score, keep-earliest-date, keep-most-complete, merge-all, flag-duplicates. If keep-highest-score is selected, [RETRIEVED_DOCUMENTS] must include score fields. Validate strategy against available document metadata.

[SOURCE_PRIORITY_RULES]

Ordered rules for resolving conflicts when sources disagree or overlap

["prefer-peer-reviewed", "prefer-more-recent", "prefer-higher-retrieval-score"]

Must be a non-empty ordered array. Each rule must reference an available metadata field or document property. Validate that referenced fields exist in [RETRIEVED_DOCUMENTS] metadata. Reject rules referencing unavailable fields.

[MAX_CITATIONS_PER_CLAIM]

Upper bound on how many sources can be cited for a single merged claim

3

Must be a positive integer between 1 and 10. Values above 10 risk citation clutter and reduced readability. Validate as integer. If null, default to 3 with a warning logged.

[CONFLICT_DISCLOSURE_MODE]

How to surface source disagreements in the merged output

"inline-annotation" or "separate-conflict-block" or "silent-prefer-highest"

Must be one of: inline-annotation, separate-conflict-block, silent-prefer-highest, omit-conflicting. For audit-heavy use cases, silent-prefer-highest requires explicit approval in the implementation checklist. Validate against compliance requirements before use.

[MIN_CITATION_CONFIDENCE]

Threshold below which citations are excluded or flagged as uncertain

0.65

Must be a float between 0.0 and 1.0. Citations with retrieval scores below this threshold are either dropped or annotated with low-confidence markers depending on [CONFIDENCE_HANDLING]. Validate as numeric. Null means no threshold filtering.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Multi-Source Citation Merging Prompt into a production RAG pipeline with validation, deduplication, and conflict resolution.

This prompt operates as a post-retrieval, pre-synthesis step in a RAG pipeline. After your retriever returns chunks from multiple documents, pass the raw retrieved passages, their source metadata, and any existing citation markers into this prompt. The model's job is to produce a deduplicated, conflict-resolved citation map before the answer generation step consumes it. Do not use this prompt as the final answer generator—it is a citation normalization and merging utility that feeds into a downstream synthesis prompt such as the RAG Citation Answer Prompt Template or the Deterministic Citation Output Contract Prompt.

Wire the prompt into your application as a middleware function between retrieval and answer generation. The function should: (1) collect all retrieved passages with their source_id, document_title, chunk_index, retrieval_score, and text fields; (2) assemble the prompt with the [RETRIEVED_PASSAGES] placeholder populated as a structured JSON array; (3) call the model with temperature=0 and response_format set to JSON to enforce the output schema; (4) validate the returned citation map against a schema that requires merged_citations (array of unique source objects), deduplication_log (which passages were collapsed and why), and conflict_flags (where sources disagree on facts, dates, or claims). If validation fails, retry once with the validation error injected into the [PREVIOUS_ERRORS] placeholder. Log every merge decision and conflict flag for auditability—this is critical for compliance workflows where you must explain why one source was preferred over another.

For high-stakes domains, insert a human review gate after citation merging when the conflict_flags array is non-empty or when the deduplication removed more than a configurable threshold of passages. Route flagged merges to a review queue with the original passages, the merged citation map, and the conflict details. Avoid the temptation to let the model silently resolve conflicts by picking a winner—surface the disagreement explicitly. When integrating with downstream answer generation, pass only the merged_citations array (not the raw passages) to keep the context window lean and prevent the answer model from re-litigating merge decisions. Test the full pipeline with overlapping document sets where you control the ground truth, and measure both citation recall (are all unique sources preserved?) and conflict detection precision (are real disagreements flagged without noise?).

IMPLEMENTATION TABLE

Expected Output Contract

Defines the exact structure, types, and validation rules for the merged citation output. Use this contract to parse the model response, validate it before downstream use, and detect malformed or incomplete citation chains.

Field or ElementType or FormatRequiredValidation Rule

merged_citations

Array of objects

Must be a non-empty array. If no citations can be merged, return an empty array and set merge_status to 'no_merge_possible'.

merged_citations[].merged_id

String, UUID v4

Must be a valid UUID v4 string. Generated by the model to uniquely identify this merged citation group.

merged_citations[].claim

String

Must be a single, self-contained factual statement. Cannot be empty or whitespace-only. Must not contain internal contradictions.

merged_citations[].source_ids

Array of strings

Must contain at least 2 source IDs from the input [SOURCE_MAP]. Each ID must exist in the provided source map. No duplicates allowed within the array.

merged_citations[].priority_source_id

String

Must be one of the IDs in source_ids. Selected based on [PRIORITY_RULES] provided in the prompt. If no priority rule applies, use the source with the earliest [DOCUMENT_DATE].

merged_citations[].conflict_flag

Boolean

Must be true if the merged sources contain contradictory details for this claim, otherwise false. When true, the conflict_note field is required.

merged_citations[].conflict_note

String or null

Required when conflict_flag is true. Must describe the specific contradiction and which source supports which variant. Must be null when conflict_flag is false.

merge_status

String enum

Must be one of: 'complete', 'partial_deduplication', 'conflict_unresolved', 'no_merge_possible'. Determines how downstream consumers should treat the merged_citations array.

PRACTICAL GUARDRAILS

Common Failure Modes

Multi-source citation merging fails in predictable ways. These are the most common production failure modes and the guardrails that catch them before users see broken attribution.

01

Duplicate Source Collapse

What to watch: The same source chunk appears under multiple retrieval IDs, and the merge prompt treats them as independent evidence, inflating apparent support for a claim. Guardrail: Pre-process retrieved chunks with content-hash deduplication before the merge step. Require the prompt to list unique source IDs and flag when multiple IDs map to identical content.

02

Priority Inversion on Conflicting Sources

What to watch: The model resolves source conflicts by picking the most fluent or recent-sounding source rather than respecting a defined priority order (e.g., authoritative document over draft, primary source over summary). Guardrail: Supply an explicit source priority schema in the prompt with document tiers, effective dates, and a conflict resolution rule. Validate that the final answer cites the higher-priority source when conflicts exist.

03

Silent Source Dropping

What to watch: The merge prompt omits a relevant source entirely because it appears redundant or the model judges it less useful, even when the source contains unique information. Users lose evidence they should see. Guardrail: Require the prompt to enumerate all input sources and explicitly mark which were used, which were excluded, and why. Add an eval check that flags answers where a provided source appears in zero citations without a documented exclusion reason.

04

Temporal Inconsistency Across Sources

What to watch: Sources with different publication dates contain contradictory facts (e.g., a policy change), and the merge prompt synthesizes them as if both are simultaneously true without surfacing the timeline conflict. Guardrail: Include source dates in the metadata passed to the merge prompt. Add an instruction: 'When sources with different dates disagree, surface the conflict explicitly and cite the most recent applicable source as current, noting the older source as historical.' Validate that date-aware conflicts produce explicit conflict notes.

05

Merge Hallucination from Insufficient Overlap

What to watch: The model is asked to merge citations from sources that address different aspects of a question, and it fabricates a synthetic claim that none of the sources individually support, stitching fragments into a false composite. Guardrail: Require the merge prompt to verify that each merged claim is independently supported by at least one source before combining. Add a post-merge check: 'For each claim in the merged answer, identify which single source provides the strongest direct support. Flag claims that rely on cross-source inference.'

06

Attribution Chain Breakage

What to watch: The merge prompt correctly cites a source chunk but loses the provenance chain back to the original document (e.g., chunk ID is present but document ID, version, or retrieval timestamp is missing). Audit trails become incomplete. Guardrail: Define a minimum citation object with required provenance fields: source_id, document_id, chunk_id, retrieval_timestamp. Validate every citation object in the merged output against this minimum contract. Reject citations with broken provenance chains.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to evaluate the quality of merged citations before shipping. Each criterion targets a specific failure mode in multi-source citation merging. Run these checks on a golden dataset of 20-50 examples with known source conflicts, duplicates, and priority rules.

CriterionPass StandardFailure SignalTest Method

Deduplication Accuracy

All duplicate citations across sources are merged into a single canonical reference. No source appears twice for the same claim.

Same source ID or URL appears in multiple citation slots for one claim. Merged output contains redundant entries.

Compare citation list length against unique source count in ground truth. Flag if output citations > unique sources.

Source Priority Adherence

When sources conflict, the higher-priority source is cited. Priority rules from [PRIORITY_RULES] are followed exactly.

Lower-priority source cited instead of higher-priority one. Both conflicting sources cited without resolution.

Inject known priority conflicts into test set. Verify cited source matches expected winner per [PRIORITY_RULES].

Conflict Disclosure Completeness

Every unresolved source conflict is surfaced with both sources cited and a conflict note. No silent suppression of disagreement.

Conflicting evidence merged into single claim without disclosure. Only one side of a conflict appears.

Audit output for conflict markers. Cross-reference with ground-truth conflict map. Flag missing conflict disclosures.

Citation Chain Integrity

Every merged citation traces back to its original source document. No orphaned citations or broken reference chains.

Citation references a source not present in [SOURCE_DOCUMENTS]. Merged citation loses original provenance.

Validate every output citation ID against [SOURCE_DOCUMENTS] index. Flag any reference not found in input set.

Merge Completeness

All relevant sources for each claim are included in the merged citation. No source that supports a claim is dropped during merge.

Source present in input documents is absent from merged output for a claim it supports. Evidence gap introduced.

Compare supporting source sets per claim between input and output. Calculate recall: cited sources / all supporting sources.

Temporal Conflict Handling

When sources have different dates, the most recent authoritative source is preferred. Stale sources are noted if cited.

Outdated source cited as primary without date caveat. Conflicting dates ignored in merge decision.

Include sources with known date conflicts in test set. Check that date-aware priority logic fires correctly.

Output Schema Compliance

Merged citations conform exactly to [OUTPUT_SCHEMA]. All required fields present. No extra fields. Types match schema.

Missing required field in citation object. Extra field not in schema. Wrong type for field value.

Validate output against [OUTPUT_SCHEMA] with JSON Schema validator. Flag any schema violations.

No Fabricated Citations

Zero citations reference sources not present in [SOURCE_DOCUMENTS]. No hallucinated source IDs, URLs, or titles.

Citation contains source ID, title, or URL not found in any input document. Model invents a source.

Exact-match all citation identifiers against input source list. Any unmatched reference is a hard failure requiring human review.

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base prompt and a single document pair. Remove strict output schema requirements initially. Use a simple format: [Claim] (Source: [DOC_ID], Priority: [PRIORITY]). Focus on getting the merge logic right before adding validation layers.

code
Merge citations from these sources. If sources conflict on the same claim, keep the one with higher priority.

Sources:
[DOCUMENTS]

Claims to merge:
[CLAIMS]

Watch for

  • Duplicate claims surviving the merge when text differs slightly
  • Priority rules being ignored when sources have similar timestamps
  • The model inventing priority rankings when none are provided
  • Missing source attribution on merged claims
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.