Partial Streaming with Source Citation Prompt

This prompt is designed for retrieval-augmented generation (RAG) systems that stream answers to users in real time while citing sources. Instead of waiting for a complete response, the model emits partial text tokens interleaved with citation index markers. A frontend can parse these markers to render clickable source links as the answer materializes. Use this when you need grounded, verifiable answers delivered progressively in chat interfaces, research assistants, or compliance-heavy domains where every claim must be traceable to evidence before the user finishes reading.

The ideal user is a product engineer or AI platform developer building a customer-facing chat interface, an internal research tool, or a compliance review dashboard. The required context includes a set of pre-retrieved evidence chunks, each with a unique index, and a user query that demands factual synthesis. This prompt is not appropriate for creative writing, open-ended brainstorming, or tasks where source grounding is unnecessary. It also should not be used when the frontend cannot parse inline citation markers, when the evidence set is too large to fit in context, or when the model's base tendency to hallucinate citations outweighs the benefits of streaming. In high-risk domains such as healthcare or legal review, pair this prompt with a human approval step and an automated citation-verification eval that checks every emitted index against the provided evidence set.

Before implementing, confirm that your streaming infrastructure supports token-by-token delivery and that your frontend can handle partial citation markers without breaking rendering. Test with edge cases: evidence chunks that contradict each other, queries with no relevant evidence, and very long answers that push token limits. If your use case requires structured JSON output rather than prose with inline citations, use a schema-first streaming prompt from the Structured Output pillar instead. If you need the model to retrieve evidence itself rather than receiving pre-retrieved chunks, combine this prompt with a tool-use or retrieval-query-rewriting prompt from the RAG pillar.

This prompt template is designed for a Retrieval-Augmented Generation (RAG) system that streams an answer to the user while simultaneously emitting inline citation markers. The core challenge is maintaining a one-to-one correspondence between generated claims and their supporting evidence chunks, all while the output is still being formed. The template uses a structured output format that interleaves text fragments with citation indices, allowing a frontend to progressively render both the answer text and clickable source links without waiting for the entire response to complete.

codeCopy
SYSTEM:
You are a precise research assistant. Your task is to answer the user's question using ONLY the provided [CONTEXT] chunks. You must cite your sources inline.

Your response must be a stream of JSON objects, each on a new line, conforming to this exact structure:
{"type": "text", "content": "<string of answer text>"}
{"type": "citation", "indices": [<array of integers>]}

RULES:
1.  Every factual claim or direct piece of information must be immediately followed by a citation object containing the index (or indices) of the [CONTEXT] chunk(s) that support it.
2.  The 'indices' array must contain integers that correspond to the 0-based index of the chunk in the provided [CONTEXT] list.
3.  If multiple chunks support a single claim, include all relevant indices in the array (e.g., [0, 2]).
4.  If the provided [CONTEXT] does not contain the answer, emit a single text object stating that the information is not available and do not emit any citation objects.
5.  Do not output any text outside of these JSON objects. The entire stream must be a sequence of valid JSON lines (NDJSON).
6.  Begin with a text object. End with a text object.

[CONTEXT]:
[CHUNK_0]: "The Apollo 11 mission landed on the Moon on July 20, 1969."
[CHUNK_1]: "Neil Armstrong was the commander of Apollo 11 and the first person to walk on the Moon."
[CHUNK_2]: "Buzz Aldrin joined Armstrong on the lunar surface about 19 minutes later."
[CHUNK_3]: "The Apollo program was run by NASA, the United States' space agency."

[INPUT]:
Who walked on the moon first during Apollo 11, and when?

To adapt this template for your own application, replace the example [CONTEXT] chunks with the actual results from your retrieval pipeline, ensuring each chunk is clearly indexed. The [INPUT] placeholder should be replaced with the user's query. The critical adaptation is in the frontend parser: you must build a streaming NDJSON parser that reads each line, parses the JSON, and updates the UI state. When a 'text' object arrives, append its content to the answer block. When a 'citation' object arrives, attach a clickable footnote marker to the preceding text segment using the provided indices to fetch source metadata. For high-stakes applications, implement a client-side validation check that flags any citation index which is out of bounds of the original [CONTEXT] array, as this indicates a model hallucination.

This prompt is designed for a streaming RAG pipeline where the model emits answer text interleaved with citation indices (e.g., [1], [2]) as chunks arrive. The application layer must handle three concurrent responsibilities: forwarding text chunks to the client UI for progressive rendering, accumulating citation indices to fetch source metadata, and validating that each citation maps to a real retrieved chunk. Do not treat this as a fire-and-forget generation call—the harness must track citation-to-chunk alignment throughout the stream.

Streaming integration pattern: Use a server-sent events (SSE) or WebSocket endpoint that accepts a session_id, the user query, and a pre-retrieved [CONTEXT] array of chunks with unique chunk_id values. The prompt template expects [CONTEXT] formatted as numbered entries (e.g., [1] chunk_id:abc123 Content: ...). On the server side, open a streaming completion call with stream: true and process each delta. Maintain a citation_map accumulator: when the model emits a citation marker like [3], record the chunk index and the surrounding text span. Before forwarding the chunk to the client, strip or annotate citation markers depending on whether your UI renders them inline or as footnotes. Log every chunk with a sequence_number and timestamp for trace debugging.

Validation and retry logic: After the stream completes (or on early termination), run a post-stream validation pass. Check that every citation index emitted falls within the range of provided context chunks. If a citation references a non-existent index, flag the response for repair. For high-stakes applications, implement a two-stage validation: first, a lightweight regex check on the accumulated text for citation format validity; second, an LLM-as-judge eval that verifies each cited chunk actually supports the claim it's attached to. If validation fails, trigger a retry with the original context plus the validation error message appended as a [CONSTRAINTS] update, instructing the model to correct or remove unsupported citations. Cap retries at two attempts before escalating to a human review queue.

Model choice and tool integration: This prompt works best with models that have strong instruction-following for structured output during streaming—Claude 3.5 Sonnet and GPT-4o are reliable starting points. Avoid models with weak citation discipline or tendency to hallucinate source indices. If your RAG pipeline uses a reranker or evidence selection step before generation, pass only the top-k chunks that meet a relevance threshold into [CONTEXT] to reduce citation noise. For production observability, instrument the harness to emit metrics: citation coverage rate (percentage of claims with a citation), hallucinated citation count, average latency per chunk, and validation pass/fail rate. Wire these into your existing monitoring stack (Prometheus, Datadog, or similar) with alerts on citation failure spikes.

What to avoid: Do not assume the model will always emit citations in numeric order or that every sentence will carry a citation. The harness must handle gaps gracefully—missing citations on low-risk statements are acceptable; missing citations on factual claims are not. Do not cache or reuse citation mappings across different queries, as chunk indices are context-specific. Finally, never present uncited claims as grounded without running the post-stream validation pass. If your use case involves regulated content (medical, legal, financial), require human approval on any response where citation coverage falls below a defined threshold before the answer reaches the end user.