Evidence Gap Report Generation Prompt

This prompt is designed for AI operations teams and knowledge-base maintainers who need to systematically identify what evidence is missing from their retrieval pipeline. It generates a structured gap report that documents missing facts, explains why those gaps degrade answer quality, and recommends specific remediation actions. Use this prompt when you are auditing retrieval quality, planning knowledge-base expansion, or building continuous improvement workflows for RAG systems. The output is not an answer to a user query; it is an internal-facing diagnostic report for the team that owns the retrieval pipeline.

To use this prompt effectively, you must provide the original user query and the full set of retrieved passages or context that was available to the model. The prompt works best when the retrieval set has already been filtered for relevance, as it focuses on identifying what is missing rather than what is irrelevant. You should also specify the expected output schema, such as a structured JSON report with gap categories, severity ratings, and remediation actions. For high-stakes domains like healthcare or finance, always route the generated gap report through a human review step before committing to knowledge-base changes or pipeline reconfiguration.

Do not use this prompt when you need a real-time answer to a user query, when you are evaluating the relevance of retrieved passages, or when you need a binary answerable/unanswerable decision. For those tasks, use the RAG Sufficiency Gate Prompt or the Context Adequacy Binary Classifier Prompt instead. This prompt is specifically for post-hoc diagnostic analysis and continuous improvement workflows. Avoid running it on every query in production; it is designed for sampling, auditing, and periodic quality reviews where the cost of a detailed report is justified by the operational insight gained.

This template generates a structured Evidence Gap Report from a user query and a set of retrieved passages. It is designed to be called after retrieval but before answer generation, acting as a diagnostic gate. The prompt forces the model to identify what information is missing, explain why that missing information matters for answer quality, and recommend concrete remediation actions for the retrieval pipeline or knowledge base maintainers. Use this when you need to move beyond a simple 'I don't know' and produce actionable documentation for continuous improvement of your RAG system.

textCopy
You are an evidence quality auditor for a RAG (Retrieval-Augmented Generation) system. Your task is to analyze the provided context against the user's query and produce a structured Evidence Gap Report.

## INPUTS
**User Query:** [USER_QUERY]
**Retrieved Context:** [RETRIEVED_CONTEXT]
**Knowledge Base Scope:** [KNOWLEDGE_BASE_DESCRIPTION]

## TASK
1.  **Decompose the Query:** Break the [USER_QUERY] into discrete information needs.
2.  **Map Evidence:** For each information need, identify which parts of the [RETRIEVED_CONTEXT] address it, partially address it, or fail to address it.
3.  **Identify Gaps:** List specific facts, entities, data points, or document types that are missing from the [RETRIEVED_CONTEXT] and are required to fully answer the query.
4.  **Assess Impact:** For each gap, explain how its absence would degrade the quality, accuracy, or completeness of a generated answer.
5.  **Recommend Remediation:** Propose concrete actions to fill each gap. These actions should be one of three types:
    *   **Query Rewrite:** A new search query to retrieve the missing information.
    *   **Knowledge Base Update:** Specific content, documents, or metadata that should be added to the [KNOWLEDGE_BASE_DESCRIPTION].
    *   **Clarification Question:** A question to ask the user to resolve an ambiguity.

## CONSTRAINTS
- Do not answer the [USER_QUERY]. Only report on evidence gaps.
- Base all analysis strictly on the [RETRIEVED_CONTEXT] provided. Do not use external knowledge.
- If no gaps exist, state that the evidence is sufficient and explain why.
- If the query is ambiguous, identify the ambiguity as a gap and recommend a clarification question.

## OUTPUT_SCHEMA
Respond with a single JSON object conforming to this schema:
{
  "query_decomposition": [
    {
      "information_need": "string",
      "status": "fully_addressed | partially_addressed | not_addressed",
      "supporting_context_ids": ["string"]
    }
  ],
  "evidence_gaps": [
    {
      "gap_id": "string",
      "missing_information": "string",
      "severity": "critical | high | medium | low",
      "impact_on_answer": "string",
      "remediation": {
        "action_type": "query_rewrite | knowledge_base_update | clarification_question",
        "action_content": "string"
      }
    }
  ],
  "overall_sufficiency_assessment": "sufficient | insufficient | partially_sufficient",
  "summary": "A concise executive summary of the findings."
}

To adapt this template, start by populating the [USER_QUERY] and [RETRIEVED_CONTEXT] placeholders with live data from your application. The [KNOWLEDGE_BASE_DESCRIPTION] is a static string that describes the scope of your indexed documents (e.g., 'Internal product documentation for Acme Corp's Widget API'). For high-stakes applications, integrate a post-generation validation step that checks the JSON schema and verifies that the supporting_context_ids in the output actually exist in the input context. This prevents hallucinated citations within the gap report itself. If the overall_sufficiency_assessment is insufficient, your orchestration layer should route the request to a human review queue or a clarification flow, rather than proceeding to answer generation.

The Evidence Gap Report prompt is designed to run as an asynchronous, post-retrieval analysis step, not as a real-time user-facing generation. It should be triggered by a monitoring job, a scheduled pipeline, or a manual review queue after a batch of queries has been processed. The primary consumer is an AI operations engineer or knowledge-base maintainer, so the output must be logged to an observability platform, written to a database, or rendered in a dashboard—not streamed directly to an end user. The prompt requires a query, the retrieved context set, and any existing answer or confidence score as inputs. Without these, the gap analysis will be speculative and unactionable.

To wire this into an application, wrap the prompt in a function that accepts a query_id, query_text, retrieved_passages (list of objects with content, source_id, and score), and an optional generated_answer. The function should construct the prompt by injecting these values into the [QUERY], [RETRIEVED_CONTEXT], and [GENERATED_ANSWER] placeholders. Before calling the model, validate that the retrieved context is not empty; if it is, log a NO_EVIDENCE event and skip the gap analysis. After receiving the model response, parse the structured JSON output and validate it against a schema that requires missing_evidence (array), impact_assessment (string), and remediation_actions (array). If parsing fails, retry once with a stricter output format instruction. Log the validated report to your observability stack with the query_id and a timestamp. For high-stakes knowledge bases (e.g., medical, legal, financial), route the report to a human review queue before any automated remediation is triggered.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities, such as gpt-4o or claude-3.5-sonnet. Set temperature to 0.0 or near-zero to maximize consistency across runs. If you are processing a large backlog of queries, batch the requests and use prompt caching for the static system instructions to reduce latency and cost. Do not use this prompt as a real-time guardrail before answer generation; it is too slow and verbose for that. Instead, pair it with a lightweight sufficiency classifier (see the RAG Sufficiency Gate Prompt) that runs inline, and reserve the Evidence Gap Report for offline analysis and continuous improvement of your retrieval pipeline or knowledge base. The remediation actions in the output—such as 'add documentation for X' or 're-index source Y'—should be fed into your knowledge-base update workflow, not executed automatically without review.