Evidence Gap Detection Prompt for RAG Abstention

This prompt is for AI engineers building Retrieval-Augmented Generation (RAG) systems where a wrong answer carries a higher cost than no answer. Its job is not to generate a response, but to act as a pre-generation gate. It takes a user query and a set of retrieved context chunks and produces a structured 'gap report' that explicitly maps which parts of the query are supported by the provided evidence and which parts are not. The ideal user is an engineering lead or senior developer who needs to programmatically decide whether to proceed with answer synthesis, trigger a second retrieval pass with rewritten queries, or escalate to a human operator. You should use this prompt when your application operates in a high-stakes domain—such as clinical informatics, legal research, or financial compliance—where hallucination is unacceptable and the system must demonstrate a clear chain of evidence before speaking.

To use this prompt effectively, you must provide a specific user query and the raw, unsummarized text chunks retrieved from your vector store or search index. The prompt is designed to work as a deterministic filter in your inference pipeline: it expects a strict output schema that your application code can parse to make a binary or tiered routing decision. For example, if the gap report indicates that the context lacks information on a specific drug interaction, your harness can immediately block answer generation and route the query to a drug database API instead. Do not use this prompt as a general-purpose confidence scorer or to produce a numeric probability. It is a qualitative reasoning tool that produces a map of evidence coverage and missing entities, not a calibrated score. Using it for numeric confidence will lead to brittle, uncalibrated results.

Avoid using this prompt when the cost of abstention is higher than the cost of a partially correct answer, such as in casual chatbots or creative brainstorming tools. It is also a poor fit for real-time, low-latency applications where the overhead of an extra LLM call before every answer is prohibitive. The next step after reading this section is to examine the prompt template and adapt the output schema to match your routing logic. Before deploying, you must build a validation harness that checks the gap report against a golden dataset of queries with intentionally insufficient context to ensure the model correctly identifies missing information without hallucinating gaps where evidence actually exists.

The prompt below is designed to be dropped into a RAG pipeline immediately before answer synthesis. It forces the model to act as an auditor, not an answer engine. The template accepts retrieved context and a user question, then outputs a structured JSON report that downstream code can parse to decide whether to answer, retrieve more, escalate, or refuse. Every placeholder is enclosed in square brackets—replace them with your actual inputs before sending the request.

textCopy
You are an evidence auditor for a retrieval-augmented generation system. Your only job is to assess whether the provided context can address the user's question. Do not answer the question. Do not speculate beyond the context. Do not fabricate missing information.

## INPUT
User Question:
[USER_QUESTION]

Retrieved Context:
[RETRIEVED_CONTEXT]

## OUTPUT SCHEMA
Return a single JSON object with exactly these fields:
{
  "coverage_assessment": "FULL" | "PARTIAL" | "NONE",
  "addressed_aspects": ["list of specific claims or sub-questions the context CAN address, with exact quotes where possible"],
  "gap_aspects": ["list of specific claims or sub-questions the context CANNOT address"],
  "missing_entities": ["list of key entities, dates, figures, or concepts mentioned in the question but absent from the context"],
  "evidence_quality_notes": "brief assessment of source recency, authority, and internal consistency within the provided context",
  "recommendation": "ANSWER" | "RETRIEVE_MORE" | "REFUSE"
}

## CONSTRAINTS
- If the context is completely irrelevant to the question, set coverage_assessment to "NONE" and recommendation to "REFUSE".
- If the context partially addresses the question but leaves material gaps, set coverage_assessment to "PARTIAL" and recommendation to "RETRIEVE_MORE".
- Only set recommendation to "ANSWER" when coverage_assessment is "FULL" and all sub-questions are directly supported by explicit evidence.
- Never invent entities, dates, or facts to fill gaps. If something is missing, list it in missing_entities.
- Quote the context verbatim in addressed_aspects when showing support.
- If multiple sources conflict, note this in evidence_quality_notes and do not resolve the conflict.

Adaptation guidance: Replace [USER_QUESTION] with the end user's query and [RETRIEVED_CONTEXT] with your retrieval pipeline's output—typically concatenated chunks with source identifiers. If your system uses multiple retrieval rounds, wire this prompt after the first retrieval pass and use the recommendation field to trigger a second retrieval with expanded or rewritten queries. For regulated domains, add a [DOMAIN_RULES] placeholder containing jurisdiction-specific abstention triggers (e.g., "Never assess coverage for drug interaction questions"). The output schema is intentionally flat to simplify downstream parsing; avoid nesting unless your validator explicitly supports nested object traversal. Always validate the JSON structure before acting on the recommendation—malformed outputs should trigger a retry or fallback to a safe refusal.

This prompt is designed to operate as a pre-generation validation step in a RAG pipeline. After your initial retrieval call returns a set of documents, you call the model with this prompt, the user's original query, and the retrieved context. The model's job is not to answer the question but to produce a structured gap report. This report identifies which aspects of the query are covered by the context and which are not. The primary output is a JSON object containing covered_aspects, gap_aspects, and retrieval_suggestions. The presence of any items in gap_aspects is the signal to halt answer generation and initiate a corrective workflow.

To integrate this into an application, implement a post-retrieval router. After parsing the model's JSON output, check the gap_aspects array. If it is non-empty, do not proceed to the answer generation prompt. Instead, route the retrieval_suggestions to a re-retrieval module. This module should use the suggestions to rewrite the query or fetch additional documents from your knowledge base. After re-retrieval, concatenate the new context with the original context and call this gap detection prompt again. Implement a maximum retry limit of 2 re-retrieval attempts. If gaps persist after 2 retries, escalate the query, the final gap report, and the full context to a human review queue rather than generating a potentially incorrect answer.

A critical failure mode for this prompt is hallucinated gaps—the model claiming an aspect is missing when the context actually contains the information. You must implement a validator that cross-references each claimed gap against the provided context. A simple approach is to use a secondary, fast model call for each gap_aspect with a strict prompt: 'Is the information described by [GAP_ASPECT] present in the following context? Answer only YES or NO.' If the validator returns YES for any claimed gap, log a hallucinated_gap_detected warning, remove the false gap from the report, and proceed with the answer generation step. Log the full gap report, the validator results, and the final routing decision (proceed, re-retrieve, or escalate) for audit trails and to monitor the health of your retrieval pipeline over time.