Evidence Sufficiency Gate Prompt Before Answer Release

This prompt operates as a programmatic gatekeeper inside your RAG pipeline, positioned after answer generation but before the response is delivered to the end user. Its job is to evaluate whether the retrieved evidence sufficiently supports the generated answer and to produce a structured gate decision: release the answer to the user, hold it for human review, or escalate it as a potential hallucination risk. Use this when you need an auditable, automated checkpoint that prevents ungrounded or speculative answers from reaching production traffic. The ideal user is a production RAG architect or AI platform engineer who already has a retrieval step, an answer generation step, and now needs a control layer between generation and delivery.

This is not a refusal prompt for the user. It is an internal pipeline control that your application consumes before deciding what to surface. The prompt expects the original user query, the retrieved evidence set, and the generated answer as inputs. It returns a structured decision object with a gate action, a sufficiency score, and a rationale. Your application code reads this decision and routes accordingly: release the answer, queue it for human review, or trigger a fallback response. This separation of concerns—gate logic in the prompt, routing logic in the application—keeps the system testable and auditable. You can log every gate decision, measure false-release rates, and tune the sufficiency threshold without touching the answer generation prompt.

Do not use this prompt when you lack a retrieval step, when the evidence set is empty by design, or when the answer is generated from parametric knowledge alone. It is also the wrong tool for user-facing refusal messages—use the sibling playbook Evidence Insufficiency Refusal Prompt Template for that. If your latency budget is extremely tight, consider running the gate asynchronously on a sample of traffic rather than blocking every response. For high-risk domains such as healthcare or legal, always pair this gate with human review for any decision that is not a clear release, and ensure the gate rationale is preserved in audit logs. Start by running the gate in shadow mode on production traffic to calibrate your sufficiency threshold before enabling blocking behavior.

This template acts as a release gate before any generated answer reaches an end user. It forces the model to assess the provided evidence against the original query and the candidate answer, producing a structured decision that your application can parse and act on. The prompt is designed to be dropped into a post-generation validation step in your RAG pipeline, after answer generation but before the response is returned to the user. It does not generate the answer itself; it judges whether the answer that was already generated should be released, held for review, or escalated.

textCopy
You are an evidence sufficiency auditor for a production RAG system. Your job is to evaluate whether the provided evidence is sufficient to support the candidate answer for the given user query. You do not generate answers. You only produce a structured gate decision.

## INPUTS

**User Query:**
[USER_QUERY]

**Candidate Answer:**
[CANDIDATE_ANSWER]

**Retrieved Evidence Passages:**
[EVIDENCE_PASSAGES]

**Evidence Sufficiency Threshold:** [SUFFICIENCY_THRESHOLD]
**Domain Risk Level:** [RISK_LEVEL]

## INSTRUCTIONS

1. Compare every factual claim in the Candidate Answer against the Retrieved Evidence Passages.
2. Identify any claims that are unsupported, contradicted, or only partially supported by the evidence.
3. Assess whether the evidence as a whole is sufficient to justify releasing the answer, given the Evidence Sufficiency Threshold and Domain Risk Level.
4. Produce a structured gate decision using the exact output schema below.

## OUTPUT SCHEMA

Return a single JSON object with these fields:
- "gate_decision": "release" | "hold" | "escalate"
- "sufficiency_score": 0.0 to 1.0
- "supported_claims_count": integer
- "unsupported_claims_count": integer
- "contradicted_claims_count": integer
- "gaps": [array of strings describing specific evidence gaps]
- "rationale": string explaining the decision in 2-3 sentences
- "recommended_action": string with concrete next step

## DECISION RULES

- "release": sufficiency_score >= threshold AND no contradicted claims AND no critical gaps
- "hold": sufficiency_score below threshold OR minor unsupported claims present, but no contradictions
- "escalate": any contradicted claims OR critical gaps in safety-relevant claims OR risk_level is "high" and sufficiency_score < 0.9

## CONSTRAINTS

- Do not fabricate evidence or fill gaps with your own knowledge.
- If the evidence is silent on a claim, mark it as unsupported.
- If evidence contradicts a claim, mark it as contradicted and escalate.
- For high-risk domains, prefer escalation when uncertain.
- Return ONLY the JSON object. No preamble, no commentary.

To adapt this template, replace the square-bracket placeholders with your pipeline values. [USER_QUERY] and [CANDIDATE_ANSWER] come from your RAG pipeline. [EVIDENCE_PASSAGES] should be the full set of retrieved passages used to generate the answer, with source identifiers if available. [SUFFICIENCY_THRESHOLD] is a float you tune based on your domain's tolerance for unsupported claims—start at 0.7 for general use, 0.9 for regulated domains. [RISK_LEVEL] should be set to "low," "medium," or "high" based on your application context. The output schema is designed for direct parsing by your application; wire the gate_decision field into your release logic so that "release" returns the answer to the user, "hold" queues it for human review, and "escalate" blocks the response and triggers an alert. Test this prompt against a golden set of known-sufficient and known-insufficient evidence-answer pairs before deploying, and monitor the false-release rate in production.

The Evidence Sufficiency Gate Prompt is not a standalone chat interaction; it is a synchronous decision node inside a retrieval-augmented generation pipeline. Its job is to inspect the retrieved context and the user query, then emit a structured gate decision—release, hold, or escalate—before the answer generation model ever sees the prompt. This means the gate must execute within your answer-generation latency budget, typically adding 200–800 ms on top of retrieval. Deploy it as a dedicated call to a fast, instruction-tuned model (e.g., GPT-4o-mini, Claude 3.5 Haiku, or a fine-tuned small model) rather than reusing your primary answer-generation model, which may be too slow or expensive for a pre-flight check.

Wire the gate prompt into your pipeline immediately after retrieval and chunk assembly. The input to the gate is the user query, the ranked retrieval results, and any metadata you have about source recency or authority. The output must be parsed into a strict schema: { "decision": "release|hold|escalate", "sufficiency_score": 0.0-1.0, "rationale": "string", "gaps": ["string"] }. Implement a JSON validator that rejects malformed gate outputs and retries once with a repair prompt before defaulting to escalate. Log every gate decision with the query hash, retrieval set ID, decision, score, and rationale for offline evaluation. For high-risk domains, route all escalate decisions to a human review queue with a pre-formatted context packet that includes the query, retrieved passages, and the gate's gap analysis.

Build a lightweight evaluation loop that runs offline: sample gate decisions, have domain experts label whether the gate should have released or held, and compute precision and recall for hold and escalate decisions. Monitor the false-release rate—cases where the gate said release but the answer contained unsupported claims—as your primary production metric. Set a latency budget alarm if gate model calls exceed 500 ms at p95. Avoid the temptation to make the gate prompt do the answering; its only job is sufficiency assessment. If you need to explain gaps to users, route the gate's gaps output to a separate refusal-generation prompt rather than asking the gate to produce user-facing text.