Gap-Aware Answer Synthesis Prompt

This prompt is for RAG application developers building user-facing answer generation systems. Use it when you need the model to synthesize an answer from retrieved evidence while explicitly distinguishing grounded claims from unsupported inferences. The prompt instructs the model to produce an answer with inline gap annotations, express calibrated uncertainty, and never fabricate information to fill holes in the evidence. This is not a general-purpose Q&A prompt. It belongs downstream of retrieval and upstream of any human review or final user delivery. It assumes you have already retrieved a set of evidence passages and need a transparent, auditable synthesis that makes evidence boundaries visible to both the system operator and the end user.

The ideal user is an AI engineer or product developer who has already built a retrieval pipeline and needs a controlled generation step that refuses to hallucinate. Required context includes a user query and a set of retrieved evidence passages. The prompt is designed for high-stakes domains where a confident wrong answer is worse than a refusal. Do not use this prompt when you lack a retrieval step, when the evidence set is empty, or when you need a fast, low-latency response without auditability. It is also inappropriate for creative writing, open-ended brainstorming, or tasks where evidence grounding is not a requirement.

Before deploying, pair this prompt with an evaluation harness that checks for faithfulness, gap detection accuracy, and appropriate uncertainty calibration. Run regression tests against a golden dataset of queries with known evidence gaps to ensure the model does not drift into overconfidence. If the output will be shown directly to end users without review, implement a confidence threshold gate that routes low-confidence syntheses to a human reviewer. The next section provides the copy-ready prompt template you can adapt and integrate into your RAG pipeline.

The following prompt template is designed to be copied directly into your prompt layer. It instructs the model to generate an answer that is faithful to the provided evidence, annotates any gaps in that evidence, and clearly separates what is supported from what is inferred. Replace each square-bracket placeholder with your application's specific data before sending the request to the model. This template assumes you have already retrieved a set of passages and performed a preliminary sufficiency check; it focuses on the synthesis step.

textCopy
You are an evidence-grounded answer synthesizer. Your task is to generate an answer to the user's query using ONLY the provided evidence passages. You must not use any outside knowledge.

## INPUT

**User Query:** [USER_QUERY]

**Retrieved Evidence Passages:**
[EVIDENCE_PASSAGES]

## OUTPUT INSTRUCTIONS

Generate a response in the following structured format:

### Answer
Synthesize a clear, concise answer to the query using only the provided evidence. If the evidence is insufficient to fully answer the query, provide the best partial answer possible.

### Grounded Claims
- For each factual statement in your answer, provide a direct quote from the evidence that supports it. Format as: `[Claim]` → `[Source Quote]`
- If a claim is a logical inference drawn from multiple pieces of evidence, mark it as `[INFERRED]` and explain your reasoning.

### Identified Gaps
List any aspects of the user's query that could not be addressed due to missing or insufficient evidence. For each gap, specify:
- **Missing Information:** What specific fact, data point, or context is absent.
- **Impact:** How this gap affects the completeness or confidence of the answer.

### Uncertainty Expression
Provide a calibrated statement about the overall confidence in the answer. Use one of the following levels and justify your choice:
- **High Confidence:** All claims are directly and unambiguously supported by the evidence.
- **Moderate Confidence:** Most claims are supported, but some rely on inference or the evidence has minor ambiguities.
- **Low Confidence:** Significant gaps exist, or the evidence is contradictory, making the answer speculative.

## CONSTRAINTS
- Do not fabricate, assume, or hallucinate any information not present in the evidence.
- If the evidence is completely irrelevant to the query, state that clearly and do not attempt an answer.
- Maintain a neutral, factual tone.

To adapt this template, start by wiring your retrieval pipeline's output into the [EVIDENCE_PASSAGES] placeholder. Format each passage with a unique identifier (e.g., [1], [2]) so the model can reference sources in the Grounded Claims section. The [USER_QUERY] should be the raw, unmodified user input. For production use, you should validate the model's output against the expected schema before surfacing it to users. A post-generation validation step should check that every claim in the Grounded Claims section has a corresponding quote and that the quotes actually appear in the provided evidence. If the validation fails, trigger a retry or escalate for human review, especially in high-stakes domains like healthcare or finance where unsupported claims carry significant risk.

The Gap-Aware Answer Synthesis prompt is designed to sit downstream of retrieval and upstream of the user-facing response surface. In a typical RAG pipeline, you retrieve a set of candidate passages, optionally re-rank them, and then pass the top-k results into this prompt alongside the user query. The prompt's output—an answer with inline gap annotations, grounded claims, and uncertainty markers—should not be sent directly to the user without validation. Instead, treat it as a structured intermediate payload that your application parses, validates, and potentially enriches before rendering.

Wire the prompt into your application with a pre-generation guard and a post-generation validator. Before calling the model, check that the retrieval set is non-empty and that at least one passage has a relevance score above your configured threshold. If the retrieval set is empty or all scores fall below threshold, skip the synthesis prompt entirely and route to a fallback response (e.g., 'I couldn't find enough information to answer that question'). After generation, parse the output to extract three components: the answer text, the list of gap annotations, and the grounded claim spans. Run a faithfulness check—either with a separate LLM judge prompt or a lightweight NLI model—to verify that each grounded claim is actually entailed by the cited passage. If the faithfulness score drops below your threshold, log the failure, increment a counter metric, and either retry with adjusted temperature or route to human review depending on risk level.

For model selection, use a model with strong instruction-following and structured output capabilities. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are all suitable. Avoid smaller or older models that may ignore the gap-annotation instructions or produce unstructured text. Set temperature between 0.0 and 0.3 to reduce variance in gap detection while preserving enough flexibility for natural answer phrasing. For retry logic, implement a maximum of two retries on validation failure. On the first retry, append the validator's specific failure reason to the prompt as additional context. On the second retry, lower the temperature to 0.0 and add an explicit instruction: 'The previous answer failed validation because [REASON]. Only make claims directly supported by the provided evidence.' If both retries fail, escalate to a human review queue and return a safe fallback message to the user.

Logging and observability are critical for this prompt because gap detection failures are often silent—the model produces a fluent answer that sounds correct but omits gap annotations or overstates confidence. Log the full prompt payload, the raw model output, the parsed gap annotations, the faithfulness check result, and the final rendered answer. Emit metrics for: gap annotation presence rate (what percentage of answers contain at least one gap marker), faithfulness pass rate, retry rate, and human escalation rate. Set alerts on drops in gap annotation presence or faithfulness pass rate, as these often signal retrieval degradation or prompt drift. For human review workflows, route any answer where the model claims high confidence but the faithfulness check fails, or where the gap annotations indicate a critical missing fact that contradicts the answer. Present reviewers with the original query, the retrieved passages, the generated answer, and the validator's specific concerns in a side-by-side review interface.

When integrating this prompt into a streaming response system, you face a trade-off: gap annotations and grounded claims require the full answer to be generated before they can be validated, but users expect low-latency streaming. One pattern is to stream the answer text to the user immediately while running validation in parallel, then append a correction or retraction if validation fails. A safer pattern for high-stakes domains is to buffer the full response, run validation, and only stream after validation passes. Choose based on your latency budget and risk tolerance. For tool-augmented workflows, if the gap annotations indicate a specific missing fact, you can trigger a targeted re-retrieval using the gap description as a query, then re-run the synthesis prompt with the augmented evidence set. Implement a maximum of one re-retrieval loop to avoid infinite cycles.

Before shipping, build a regression test suite with at least 20 query-evidence pairs covering: fully sufficient evidence, partially sufficient evidence, insufficient evidence, conflicting evidence, and edge cases with ambiguous queries. For each test case, define expected behavior: should the answer contain gap annotations? Should it refuse? What claims should be marked as grounded versus inferred? Run this suite on every prompt or model change and gate deployment on passing all test cases. Start with a high human-review routing rate in production and gradually tune thresholds down as you build confidence in the pipeline's gap detection accuracy.