Evidence Sufficiency Gate Prompt for RAG Outputs

This prompt acts as a pre-generation circuit breaker. Its job-to-be-done is to prevent the language model from attempting to synthesize an answer when the retrieved context is insufficient, irrelevant, or contradictory. The ideal user is an AI engineer or platform operator responsible for the factual reliability of a RAG system, particularly in high-stakes domains like healthcare, legal, finance, or compliance, where the cost of a hallucinated answer far exceeds the cost of declining to answer. You should use this gate when your retrieval pipeline is noisy, when your knowledge base has coverage gaps, or when user queries frequently fall outside the scope of your indexed documents. It is a critical component for building retrieval quality monitors and automated guardrails that run before answer generation, not after.

To use this gate effectively, you must provide it with the user's original query and the complete, unmodified set of retrieved evidence chunks (including their metadata, such as document IDs or relevance scores). The prompt does not perform retrieval itself; it assumes you have already executed your search, vector, or hybrid retrieval step. The gate's output should be a structured decision—typically a boolean or categorical label like SUFFICIENT, INSUFFICIENT, or CONTRADICTORY—along with a concise rationale. This structured output allows your application logic to branch cleanly: proceed to answer generation, trigger a re-retrieval with an expanded query, or surface a 'cannot answer' message to the user with a specific reason (e.g., 'No relevant information found' vs. 'Sources contradict each other'). In high-risk deployments, log the gate's decision and rationale alongside the final answer for auditability.

Do not use this prompt as a substitute for improving your retrieval pipeline. If your retrieval consistently returns poor results, the gate will correctly block answers, but the user experience will degrade into a system that rarely responds. The gate is a safety net, not a retrieval fix. Also, avoid using this prompt for open-ended creative tasks, chitchat, or scenarios where the model is expected to use its own parametric knowledge. The gate is designed for strict evidence-grounded Q&A. Finally, do not rely on this gate alone for factual accuracy; it checks for evidence sufficiency, not for the correctness of the evidence itself. A downstream hallucination detection or fact-verification step is still required for a complete quality assurance pipeline.

This prompt acts as a pre-generation gate in your RAG pipeline. Before your system spends tokens synthesizing an answer, it evaluates whether the retrieved context is collectively sufficient to address the user's question. The prompt forces a binary decision with a confidence score and actionable reasoning, making it suitable for automated decision points where you need to either proceed with generation, trigger re-retrieval, or escalate to a human. Use this when the cost of a wrong answer exceeds the cost of saying "I don't have enough information."

textCopy
You are an evidence sufficiency evaluator for a retrieval-augmented generation system. Your job is to determine whether the provided evidence is sufficient to answer the user's question accurately and completely. Do not answer the question itself. Only assess the evidence.

## INPUT

**User Question:**
[QUESTION]

**Retrieved Evidence:**
[EVIDENCE]

## EVALUATION CRITERIA

Assess the evidence against these criteria:
1. **Coverage:** Does the evidence address all parts of the question? If the question has multiple sub-questions, each must be covered.
2. **Specificity:** Is the evidence specific enough to produce a precise answer, or is it vague, tangential, or only partially relevant?
3. **Consistency:** If multiple evidence passages are provided, are they consistent with each other? Flag contradictions.
4. **Recency:** If the question implies a time constraint, is the evidence current enough?
5. **Authority:** Does the evidence come from a source that should know the answer, or is it speculative, opinion-based, or from an unreliable context?

## OUTPUT FORMAT

Return a JSON object with the following schema:

```json
{
  "decision": "SUFFICIENT" | "INSUFFICIENT" | "PARTIAL",
  "confidence": 0.0-1.0,
  "reasoning": "Concise explanation of the decision, referencing specific gaps or strengths in the evidence.",
  "gaps": ["List of specific information gaps if any"],
  "contradictions": ["List of any contradictions found between evidence passages"],
  "recommended_action": "PROCEED" | "RETRIEVE_MORE" | "NARROW_QUESTION" | "ESCALATE" | "ABSTAIN"
}

CONSTRAINTS

• If the evidence is completely absent or empty, return INSUFFICIENT with confidence 1.0.
• If the evidence partially answers the question but has clear gaps, return PARTIAL with a confidence reflecting how much is covered.
• If the evidence is sufficient but contains minor inconsistencies, return SUFFICIENT but flag the contradictions.
• Do not fabricate evidence. Only assess what is provided.
• If the question is ambiguous, note this in reasoning and recommend NARROW_QUESTION.

Adapt this template by adjusting the evaluation criteria to match your domain's evidence standards. For legal or clinical use cases, add criteria for source provenance and regulatory compliance. For customer support, you might lower the specificity bar but raise the consistency requirement. The output schema can be extended with domain-specific fields such as missing_document_ids or required_retrieval_filters to feed directly into your retrieval loop. Always validate the JSON output against the schema before proceeding—malformed outputs should trigger a retry or fallback to a conservative INSUFFICIENT decision. For high-stakes domains, route PARTIAL decisions to a human reviewer rather than automatically retrieving more.

The Evidence Sufficiency Gate is not a standalone prompt; it is a decision point inserted between retrieval and answer generation. Its job is to prevent the LLM from attempting to answer when the retrieved context is too thin, off-topic, or contradictory to support a reliable response. In practice, this means the gate prompt receives the user's question and the raw retrieval results, then returns a structured verdict that your application code can act on before spending tokens on a full answer generation call. The primary integration points are your retrieval pipeline's post-processing step and your answer generation router.

To wire this into a production RAG system, place the gate after retrieval deduplication and re-ranking but before the final context assembly for the answer prompt. The application should call the gate model with the [QUESTION] and [RETRIEVED_CHUNKS] placeholders populated. Parse the output for a sufficiency field (e.g., sufficient, insufficient, partial). On sufficient, proceed to answer generation with the same chunks. On insufficient, return a controlled abstention message to the user or trigger an expanded retrieval loop. On partial, you may choose to answer with explicit caveats or request user clarification. Log the gate verdict, confidence score, and chunk IDs for every request to build a retrieval quality dashboard. For high-stakes domains, require a confidence threshold above 0.8 before auto-proceeding; route anything below to a human review queue with the evidence and question attached.

Validation and retry logic should be explicit. If the gate model returns malformed JSON, retry once with a stricter schema reminder in the system prompt. If it fails again, default to insufficient and escalate—never guess. Implement a circuit breaker: if the gate rejects more than 60% of queries in a rolling window, pause the pipeline and alert the retrieval team. Model choice matters here; a smaller, faster model (e.g., a quantized 7B or 13B parameter model) often suffices for this classification task, keeping latency and cost low before the more expensive answer generation step. Avoid using the same model instance for both gating and answering to prevent correlated failures. Store gate decisions alongside final answers in your trace logs so you can measure whether the gate correctly predicted answer quality over time.