Evidence Gap Identification and Fill Prompt

This prompt is designed for RAG systems that have already produced an answer, but that answer contains claims that are unsupported, speculative, or entirely absent from the retrieved context. Instead of regenerating the entire answer blindly—which risks repeating the same hallucination—this prompt instructs the model to act as an auditor. Its job is to identify the specific evidence gaps in the generated answer, formulate targeted retrieval queries to fill those gaps, and produce a structured gap-to-query mapping that your application can use to execute a second, targeted retrieval round. The core job-to-be-done is evidence gap triage: you need to know exactly what evidence is missing before you can go fetch it.

Use this prompt when your initial retrieval was too narrow, when the model made an inferential leap beyond the provided documents, or when a downstream validator flags low citation coverage. It is particularly effective when the answer is mostly grounded but contains a few unsupported assertions—the auditor can isolate those assertions without discarding the valid portions of the response. The ideal user is an AI engineer or platform developer who has access to a retrieval system that accepts programmatic queries and can execute a second retrieval pass based on the auditor's output. You must provide the original question, the generated answer, and the initially retrieved context as inputs. The output is a structured JSON mapping of gap descriptions to reformulated retrieval queries, which your harness can iterate over to fetch missing evidence.

Do not use this prompt for simple formatting errors, malformed JSON, or cases where the entire answer is a hallucination with no retrievable grounding. If every claim is fabricated, there are no gaps to identify—the correct recovery is a full regeneration with stricter grounding constraints, not gap analysis. Similarly, do not use this prompt when the failure mode is a tool-call schema mismatch, an API error, or a context window overflow; those require different recovery strategies covered in the Tool Call Recovery, Infrastructure Error Retry, and Context Window Recovery playbooks. This prompt also assumes that your retrieval system can return useful results for the reformulated queries. If your corpus simply does not contain the evidence needed, the auditor will correctly identify gaps that cannot be filled, and your system should escalate to answer abstention rather than looping indefinitely.

Before wiring this into production, define a retry budget. A single audit-and-retrieve cycle is often sufficient, but if the second retrieval still leaves gaps, decide whether to audit again, fall back to a constrained regeneration, or escalate for human review. Log the gap-to-query mappings and the results of each retrieval round for observability. In high-stakes domains such as healthcare, legal, or finance, always route outputs with unresolved gaps to human review rather than presenting partially grounded answers to users. The next section provides the copy-ready prompt template you can adapt for your own RAG recovery harness.

This prompt template is the core instruction set for an evidence gap analysis and fill workflow. It is designed to be wired into a retrieval retry harness where a previously generated answer has been flagged for insufficient grounding. The prompt instructs the model to act as a forensic auditor, comparing each factual claim in the generated answer against the provided evidence context. Its primary output is a structured mapping of unsupported claims to new, targeted retrieval queries, enabling a second-pass retrieval that is far more precise than a naive query rewrite.

codeCopy
You are an evidence auditor for a retrieval-augmented generation (RAG) system. Your task is to analyze a generated answer against the provided source evidence, identify specific claims that lack sufficient support, and formulate targeted retrieval queries to fill those gaps.

## INPUT
**Generated Answer:**
[GENERATED_ANSWER]

**Provided Evidence Passages:**
[RETRIEVED_EVIDENCE]

**Original User Query:**
[USER_QUERY]

## INSTRUCTIONS
1.  **Extract Claims:** Decompose the Generated Answer into a list of discrete, verifiable factual claims. Ignore stylistic or transitional language.
2.  **Match to Evidence:** For each claim, determine if it is **SUPPORTED**, **PARTIALLY_SUPPORTED**, or **UNSUPPORTED** by the Provided Evidence Passages. A claim is only SUPPORTED if the evidence directly confirms it without requiring significant inference.
3.  **Identify Gaps:** Collect all claims marked as UNSUPPORTED or PARTIALLY_SUPPORTED. For each, write a concise description of the specific evidence gap.
4.  **Formulate Queries:** For each evidence gap, generate a single, highly specific retrieval query designed to find the missing information. The query should be a standalone search string, not a question about the answer. Prioritize precision over recall.
5.  **Output Generation:** Produce a JSON object conforming to the [OUTPUT_SCHEMA] below. Do not include any text outside the JSON object.

## CONSTRAINTS
- Do not hallucinate support. If evidence is missing, mark the claim as UNSUPPORTED.
- If the entire answer is well-supported, return an empty `evidence_gaps` array.
- Limit the total number of new queries to [MAX_QUERIES] to stay within a retry budget.
- If a claim is a direct quote, verify it is present verbatim in the evidence. Flag any deviation as UNSUPPORTED.

## OUTPUT_SCHEMA
{
  "answer_summary": "A one-sentence summary of the generated answer's main point.",
  "overall_grounding_score": 0.0, // Float between 0.0 (no support) and 1.0 (fully supported)
  "evidence_gaps": [
    {
      "unsupported_claim": "The exact text of the unsupported claim.",
      "gap_description": "A clear explanation of what evidence is missing.",
      "retrieval_query": "A precise, standalone search query to fill the gap.",
      "severity": "CRITICAL | MAJOR | MINOR"
    }
  ]
}

To adapt this template for your application, start by defining the source of [GENERATED_ANSWER] and [RETRIEVED_EVIDENCE]. The evidence should be passed as raw text chunks with their source identifiers prepended (e.g., [Source A] ...). The [MAX_QUERIES] placeholder is a critical safety valve; set it based on your latency budget for the retry loop—typically between 3 and 5. For high-stakes domains like healthcare or finance, you should route outputs with CRITICAL severity gaps to a human review queue before any new retrieval is executed. The overall_grounding_score can be used as a circuit breaker: if it falls below a threshold like 0.4, it may be more cost-effective to regenerate the answer from scratch with stricter instructions than to attempt a piecemeal fill.

This prompt is designed to sit between a failed answer validation step and a re-retrieval step in your RAG pipeline. When your citation completeness check or grounding evaluator flags an answer as having insufficient evidence, this prompt takes the original question, the generated answer, and the previously retrieved context, then produces a structured gap analysis and a set of targeted retrieval queries. The output is machine-readable JSON that your application can parse to trigger additional retrieval calls before regenerating the answer.

Wire the prompt into your pipeline as a recovery node. The input payload should include the original user query, the full generated answer that failed validation, the list of retrieved passages with their metadata (source ID, retrieval score, chunk index), and the specific validation failure reasons from your upstream evaluator. Parse the output JSON to extract the gap_queries array and feed each query into your retrieval system—vector search, keyword search, or hybrid—with the same collection scope but potentially relaxed similarity thresholds to capture broader evidence. Collect the new passages, deduplicate against the original context set, and append them to the context before calling your answer generation prompt again. Implement a retry budget: if after two gap-fill cycles the answer still fails citation completeness checks, escalate to a human reviewer or trigger an abstention response.

Add validation gates at each stage. Before executing the gap queries, validate that the prompt output contains the required fields: identified_gaps (array of objects with claim, missing_evidence_type, and search_terms), gap_queries (array of strings), and abstention_recommended (boolean). If abstention_recommended is true, skip retrieval and route directly to your abstention handler. After retrieval, run a deduplication check to avoid re-feeding the model with passages it already saw. Log every cycle—original validation failure, gap analysis output, queries executed, new passages retrieved, and regeneration result—to your observability platform so you can measure recovery success rates and identify patterns that require prompt or retrieval tuning. For high-stakes domains, insert a human review step before the regenerated answer is surfaced to the user, especially when the gap-fill cycle introduces sources with lower relevance scores than the original set.