RAG Sufficiency Gate Prompt Template

This prompt is for RAG pipeline engineers and conversational AI developers who need a structured, programmatic decision about whether retrieved evidence is sufficient to answer a user query. The job-to-be-done is preventing confident wrong answers by inserting an explicit sufficiency gate before answer generation. The ideal user is building a production RAG system where hallucination carries real cost—customer-facing support, clinical review, financial analysis, or any domain where a wrong answer is worse than no answer. They have already retrieved a set of passages and need a binary answerable/unanswerable decision, a list of specific missing information, and a calibrated confidence score that downstream routing logic can act on.

Do not use this prompt when you need a fast, lightweight check with minimal latency. This is a thorough assessment designed for high-stakes gates, not a streaming pre-filter. It is also not a replacement for answer grounding verification—this prompt assesses evidence sufficiency before an answer exists, while grounding checks verify claims after generation. If your retrieval pipeline already produces highly curated, authoritative results for a narrow domain, a simpler relevance threshold may suffice. Use this prompt when the cost of a false-positive answerability claim (generating an answer from insufficient evidence) is high enough to justify the additional inference step and latency.

The prompt requires several structured inputs to function reliably: the original user query, the full set of retrieved passages with source identifiers, and any domain-specific constraints or authority requirements. The output is a structured JSON object with a binary decision, a confidence score, an enumerated list of missing information items with severity ratings, and a rationale. Before deploying, you must calibrate the confidence threshold against your specific use case using a labeled evaluation set—do not assume a default threshold will work across domains. The companion eval harness in this playbook provides precision and recall checks for false-positive answerability claims and missed gap detection, which you should run before production release.

This prompt template implements a sufficiency gate that sits between retrieval and answer generation in a RAG pipeline. It forces the model to make an explicit binary decision about answerability before any user-facing response is produced. The template is designed to be copied directly into your prompt management system, with square-bracket placeholders that you replace with your application's specific inputs, schemas, and constraints.

textCopy
You are an evidence sufficiency evaluator for a retrieval-augmented generation system. Your job is to determine whether the provided evidence passages contain enough information to answer the user's query accurately and completely.

## INPUT

**User Query:** [QUERY]

**Retrieved Evidence Passages:**
[EVIDENCE_PASSAGES]

**Domain Context:** [DOMAIN_CONTEXT]

## OUTPUT SCHEMA

Return a JSON object with exactly these fields:

{
  "answerable": boolean,
  "confidence": number (0.0 to 1.0),
  "missing_information": [
    {
      "category": string ("factual_gap" | "temporal_gap" | "entity_gap" | "authority_gap" | "specificity_gap" | "contradiction"),
      "description": string,
      "severity": string ("critical" | "major" | "minor"),
      "search_suggestion": string or null
    }
  ],
  "evidence_coverage": {
    "fully_supported_aspects": [string],
    "partially_supported_aspects": [string],
    "unsupported_aspects": [string]
  },
  "rationale": string,
  "recommended_action": string ("proceed_to_generation" | "request_clarification" | "refuse_to_answer" | "partial_answer_with_caveats")
}

## CONSTRAINTS

[CONSTRAINTS]

## EVALUATION RULES

1. Mark as answerable ONLY if all material aspects of the query can be addressed with the provided evidence.
2. If evidence is partially sufficient but critical facts are missing, mark as unanswerable with recommended_action "request_clarification" or "refuse_to_answer".
3. If evidence contains contradictions between sources, flag this in missing_information with category "contradiction".
4. For time-sensitive queries, check whether evidence timestamps or recency indicators are present and sufficient.
5. Do not infer information not explicitly present in the evidence passages.
6. If the query requires authoritative sources and none are provided, flag this as an authority_gap.
7. Confidence should reflect the proportion of query aspects fully supported by evidence.

## EXAMPLES

[EXAMPLES]

## RISK LEVEL

[RISK_LEVEL]

To adapt this template for your system, replace the placeholders as follows: [QUERY] with the user's original question or a decomposed sub-question. [EVIDENCE_PASSAGES] with the top-k retrieved chunks, each labeled with source identifiers and retrieval scores. [DOMAIN_CONTEXT] with a brief description of the knowledge domain, expected evidence types, and any temporal requirements. [CONSTRAINTS] with additional rules specific to your use case, such as minimum source count, required authority levels, or domain-specific gap categories. [EXAMPLES] with 2-4 few-shot examples showing correct sufficiency assessments for your domain. [RISK_LEVEL] with guidance on how conservative the gate should be—use "high" for safety-critical applications where false positives are unacceptable, "medium" for most production systems, or "low" for exploratory or internal tooling. After adapting, validate the output against your application's expected schema before wiring it into the generation pipeline.

The RAG Sufficiency Gate is designed to sit between retrieval and answer generation in a RAG pipeline. It should be called after the retrieval step returns a set of passages but before those passages are passed to an answer-generation model. The gate's binary answerable decision determines whether the pipeline proceeds to answer generation, escalates for human review, or triggers a clarification request to the user. This is a pre-generation guardrail, not a post-hoc evaluation—its job is to prevent the system from attempting to answer when evidence is insufficient, which is the single most common cause of hallucination in production RAG systems.

Integration pattern: Wire the prompt as a synchronous check in your RAG orchestration layer. Pass the user query and the top-k retrieved passages (with source metadata) into the template. Parse the JSON output and branch on the answerable field. If answerable is true, route to answer generation with the same passages. If false, route to a fallback path: either a clarification prompt that uses the missing_information list to ask the user a specific follow-up question, or a refusal response that explains what's missing. Validation: Validate the output JSON against a schema that requires answerable as a boolean, confidence as a float between 0.0 and 1.0, and missing_information as an array of strings. Reject and retry (up to 2 attempts) if the schema fails. Log every gate decision with the query hash, passage IDs, answerable value, confidence score, and model used for later eval.

Model choice and latency: Use a fast, instruction-tuned model for the gate—latency matters because this is a blocking step before answer generation. GPT-4o-mini, Claude Haiku, or a fine-tuned small model are good candidates. Avoid using the same large model you use for answer generation unless latency is acceptable. Eval integration: Maintain a golden dataset of queries with known-sufficient and known-insufficient retrieval sets. Run the gate against this dataset in CI before deploying prompt changes. Track false-positive rate (gate says answerable but evidence is actually insufficient) and false-negative rate (gate says unanswerable but evidence is sufficient). False positives are more dangerous in production because they lead to hallucinated answers. Set your confidence threshold based on your application's risk tolerance—start at 0.7 and tune with production data. Human review hook: For high-stakes domains, route answerable: false decisions with confidence above 0.4 to a human review queue rather than auto-refusing, since borderline cases may contain partial evidence worth a human check.

What to avoid: Do not use the gate as a post-answer factuality check—it evaluates evidence sufficiency, not answer correctness. Do not skip validation of the gate's output JSON; malformed outputs can silently default to answerable: true in some routing logic. Do not treat the gate's confidence score as a probability in the strict statistical sense—it's a model-generated calibration signal that should be empirically evaluated against your own data. Finally, do not deploy the gate without logging; without logs, you cannot tune thresholds, detect drift, or debug false-positive spikes when retrieval quality changes.