Missing Evidence Identification Prompt

This prompt is designed for RAG system improvement workflows. Use it when you need to analyze an AI-generated answer against a set of retrieved context documents and identify which claims in the answer lack supporting evidence. The prompt distinguishes between two critical gap types: retrievable gaps, where the information likely exists in your knowledge base but was not surfaced by the current retrieval strategy, and knowledge-boundary limitations, where the required information falls outside the scope of your indexed documents. This distinction is essential for guiding retrieval optimization efforts—retrievable gaps suggest tuning chunking parameters, embedding models, or query rewriting, while knowledge-boundary gaps indicate a need to expand the document corpus itself.

Run this prompt during offline evaluation of RAG pipelines, as part of a feedback loop for tuning chunking strategies or embedding models, or before shipping a new retriever to production. The ideal user is an AI engineer or search relevance engineer who owns retrieval quality and needs structured diagnostics rather than anecdotal spot checks. You should have a set of representative queries, their corresponding retrieved context, and the AI-generated answers ready before invoking this prompt. The output is a structured gap analysis that can feed directly into retrieval metric dashboards or prioritization backlogs.

Do not use this prompt for real-time answer generation or as a substitute for a fact-verification system that requires authoritative ground truth. This prompt operates on the relationship between an answer and the context you provided—it does not access external knowledge or validate claims against the real world. It also should not replace human review when the output will be used in regulated domains such as healthcare, legal, or financial compliance. In those cases, treat the prompt's gap analysis as a prioritization aid for human reviewers, not as a final determination of factual correctness. For production monitoring of live RAG answers, pair this with a separate faithfulness evaluation prompt that runs on every response.

This template is the core instruction set for a Missing Evidence Identification workflow. It is designed to be dropped into an evaluation harness that supplies the original AI answer and the retrieved context used to generate it. The prompt instructs the model to act as a strict auditor, decomposing the answer into discrete claims and checking each one for direct support in the source material. The output is a structured report that separates retrievable gaps from knowledge-boundary limitations, giving your team a clear signal for improving retrieval precision or expanding the knowledge base.

textCopy
You are an expert evidence auditor. Your task is to analyze an AI-generated answer against the provided source context and identify every factual claim that lacks direct supporting evidence.

## INPUT
**AI Answer:**
[AI_ANSWER]

**Source Context (retrieved documents):**
[SOURCE_CONTEXT]

## INSTRUCTIONS
1. Decompose the AI Answer into a list of discrete, verifiable factual claims. Ignore opinions, stylistic phrasing, and purely structural language.
2. For each claim, search the Source Context for direct evidence that supports it. A claim is supported only if the context explicitly states the fact or provides data from which the fact is a direct, inescapable inference.
3. Classify each unsupported claim into one of two categories:
   - **Retrievable Gap:** The claim is factual in nature and likely exists in the broader knowledge base, but the provided context does not contain it. This indicates a retrieval failure.
   - **Knowledge-Boundary Limitation:** The claim requires information beyond the scope of the source material (e.g., future predictions, proprietary data, real-time events, or the model's own pre-training knowledge). This indicates a fundamental limitation of the available evidence.
4. For each Retrievable Gap, suggest a specific search query that would likely retrieve the missing evidence.

## CONSTRAINTS
- Do not mark a claim as unsupported if the source context implies it through necessary and obvious reasoning. Flag only genuinely missing evidence.
- If the AI Answer states a fact that is contradicted by the source context, mark it as a "Contradiction" instead of a "Retrievable Gap" and cite the contradicting source.
- Be precise in your citations. When a claim is supported, cite the exact source passage.

## OUTPUT_SCHEMA
Return a valid JSON object with the following structure:
{
  "answer_summary": "A one-sentence summary of the AI answer.",
  "total_claims": <integer>,
  "supported_claims": <integer>,
  "unsupported_claims": [
    {
      "claim_text": "The exact claim from the answer.",
      "gap_type": "retrievable_gap | knowledge_boundary_limitation | contradiction",
      "explanation": "Why the source context fails to support this claim.",
      "suggested_query": "A retrieval query to find the missing evidence, or null if not applicable.",
      "contradicting_source": "The exact source passage that contradicts the claim, or null."
    }
  ],
  "overall_assessment": "A brief summary of the answer's groundedness and the primary failure mode (retrieval vs. knowledge boundary)."
}

To adapt this template, replace the [AI_ANSWER] and [SOURCE_CONTEXT] placeholders with your data. The [SOURCE_CONTEXT] should be the raw text of the retrieved chunks, concatenated with clear document separators. For production use, wrap this prompt in a validation layer that parses the JSON output and confirms the total_claims count matches the length of the decomposed claims list. If the model returns malformed JSON, use a retry prompt with the schema re-stated. For high-stakes domains like healthcare or finance, always route outputs flagged as knowledge_boundary_limitation or contradiction to a human review queue before any downstream action is taken.

The Missing Evidence Identification prompt is designed to sit inside an evaluation harness that runs after a RAG system generates an answer but before that answer reaches the end user or is logged as complete. The harness calls the prompt with the generated answer and the retrieved context chunks, then parses the structured output to identify claims that lack supporting evidence. This output feeds two downstream workflows: retrieval optimization (improving what gets fetched) and answer revision (adding disclaimers or suppressing unsupported claims). The harness must treat the prompt's output as a signal, not a final verdict—false positives on missing evidence are common when the model misreads implicit support or when context chunks contain the evidence but the judge model misses it.

Wire the prompt into your application as a post-generation evaluation step. After your RAG pipeline produces an answer, extract the retrieved context passages that were actually provided to the generator model. Pass both the answer and the context array into the prompt using the [ANSWER] and [RETRIEVED_CONTEXT] placeholders. Parse the JSON output into a structured object with fields for each identified claim, its grounding status, the missing evidence description, and a retrievability flag. Validate the output schema immediately: check that required fields are present, enums match expected values, and claim text actually appears in the source answer. If validation fails, retry once with a stricter schema reminder. Log every evaluation result with a trace ID linking the answer, context, and judge output for later analysis. For high-stakes domains like healthcare or legal, route outputs flagged as having critical unsupported claims to a human review queue before any automated action is taken.

Model choice matters for this harness. Use a capable judge model—GPT-4o, Claude 3.5 Sonnet, or equivalent—because detecting subtle evidence gaps requires strong reading comprehension. Smaller or faster models will miss implicit unsupported claims and produce higher false-negative rates. If latency is critical, run the evaluation asynchronously after the answer is delivered and use the results to improve future retrievals rather than blocking the current response. Store evaluation results in a structured log with fields for answer_id, context_hash, claim_count, unsupported_count, retrievable_gap_count, and knowledge_boundary_count. Aggregate these metrics over time to detect retrieval degradation, context chunking problems, or generator model drift. Never use the missing evidence count alone as a pass/fail gate without human calibration—some answers legitimately require inference beyond the provided context, and the prompt distinguishes these knowledge-boundary cases from retrieval failures.

Build a feedback loop from this harness into your retrieval system. When the prompt identifies retrievable gaps—claims that could be supported if better context were fetched—use the missing evidence descriptions as queries for additional retrieval rounds or as signals to adjust chunking strategies, embedding models, or retrieval parameters. Track whether follow-up retrievals actually find the missing evidence to measure the prompt's precision as a retrieval diagnostic. For knowledge-boundary gaps where the evidence doesn't exist in your corpus, route the gap descriptions to your content or knowledge base team as expansion candidates. Avoid the trap of treating every unsupported claim as a retrieval failure—the prompt's distinction between retrievable and knowledge-boundary gaps is the key signal that prevents wasted engineering effort on unfixable gaps.