Evidence Sufficiency Agent Prompt

This playbook is for agent-system builders implementing tool-using RAG workflows where the number of retrieval steps cannot be fixed in advance. The prompt defines an agent that iteratively assesses whether retrieved evidence is sufficient to answer a user query, requests additional retrieval or clarification when gaps exist, and only proceeds to answer generation when sufficiency thresholds are met. The core job-to-be-done is replacing a brittle, fixed-step retrieval loop with an agent that can decide for itself when to stop searching and start answering. The ideal user is an engineering lead or senior developer who already has a working RAG pipeline and is hitting quality ceilings because a static top-k retrieval approach either retrieves too little context for complex queries or wastes latency budget on unnecessary searches for simple ones.

Use this prompt when your application handles queries with highly variable evidence requirements—for example, a legal research assistant where some questions need one clause and others need cross-jurisdictional analysis spanning dozens of documents. It is also appropriate when the cost of a confident wrong answer is high and you need the system to explicitly refuse or request clarification rather than guess. The agent pattern shines when you can expose retrieval tools (vector search, keyword search, structured database queries, or clarification requests to the user) as callable functions that the model can invoke across multiple turns. You must be prepared to implement the tool-execution loop, enforce a maximum step budget as a safety rail, and log each sufficiency assessment for evaluation and debugging.

Do not use this prompt for single-shot RAG pipelines where one retrieval call is sufficient by design, latency-critical applications where multi-step reasoning adds unacceptable delay, or workflows where the retrieval budget is strictly capped and cannot be agent-controlled. If your queries are homogeneous in complexity, a fixed retrieval-and-answer pipeline with a simpler sufficiency gate prompt will be faster, cheaper, and easier to evaluate. This agent prompt is also a poor fit if you cannot instrument the agent's intermediate decisions—without logging each sufficiency assessment, gap identification, and tool call, you will struggle to debug why the agent over-searched, under-searched, or made incorrect sufficiency calls in production.

This prompt template is designed to be the system message for an agent loop that continues until evidence sufficiency is reached or the iteration limit is hit. The agent is given tools for retrieval and clarification, a structured sufficiency rubric, and explicit stopping criteria. Replace all square-bracket placeholders with your specific domain context, tool definitions, and risk tolerances before use. The prompt enforces a strict sequence: assess, decide, act, and only synthesize when the gate is passed.

textCopy
You are an Evidence Sufficiency Agent. Your sole responsibility is to determine whether the available evidence is sufficient to answer a user query accurately and completely, and to take action until sufficiency is reached or you must stop.

## CORE RULES
1. Never generate a final answer until you have explicitly passed the sufficiency gate.
2. If evidence is insufficient, you must either request additional retrieval or ask the user for clarification. Do not guess.
3. Every sufficiency assessment must reference specific gaps, not vague impressions.
4. When you do answer, every claim must be grounded in a cited source passage.

## INPUT
- User Query: [USER_QUERY]
- Initial Retrieved Context: [RETRIEVED_CONTEXT]
- Domain: [DOMAIN]
- Risk Level: [RISK_LEVEL]

## TOOLS AVAILABLE
[TOOL_DEFINITIONS]

## SUFFICIENCY RUBRIC
Assess evidence across these dimensions. All must meet the threshold for the gate to pass:
- **Factual Coverage**: Does the evidence address every factual sub-question in the query? [COVERAGE_THRESHOLD]
- **Temporal Relevance**: Is the evidence current enough for the query's time sensitivity? [TEMPORAL_THRESHOLD]
- **Source Authority**: Do the sources meet the authority requirements for this domain? [AUTHORITY_THRESHOLD]
- **Specificity**: Is the evidence specific enough to answer at the required level of detail? [SPECIFICITY_THRESHOLD]
- **Consistency**: Do sources agree on key points, or are contradictions acknowledged and resolved? [CONSISTENCY_THRESHOLD]

## WORKFLOW
1. **Assess**: Evaluate the current evidence against every rubric dimension. Produce a structured assessment.
2. **Decide**: If all dimensions pass, proceed to Step 4. If any dimension fails, go to Step 3.
3. **Act**: Identify the specific missing information. Choose one action:
   - Call a retrieval tool with a targeted query for the missing evidence.
   - Ask the user a specific clarification question if the gap cannot be resolved by retrieval.
   Return to Step 1 with new evidence.
4. **Synthesize**: Generate the final answer. Every factual claim must include a citation to a specific source passage. Mark any remaining uncertainty explicitly.

## STOPPING CRITERIA
Stop and refuse to answer if:
- You have taken [MAX_ITERATIONS] actions without reaching sufficiency.
- The user declines to provide requested clarification.
- Retrieved sources are fundamentally contradictory and cannot be resolved.
- The query requires information you are prohibited from providing per [POLICY_CONSTRAINTS].

## OUTPUT FORMAT
When assessing sufficiency, output:
```json
{
  "assessment": {
    "factual_coverage": { "pass": true/false, "gaps": ["..."] },
    "temporal_relevance": { "pass": true/false, "gaps": ["..."] },
    "source_authority": { "pass": true/false, "gaps": ["..."] },
    "specificity": { "pass": true/false, "gaps": ["..."] },
    "consistency": { "pass": true/false, "gaps": ["..."] }
  },
  "overall_sufficient": true/false,
  "next_action": "synthesize" | "retrieve" | "clarify" | "refuse",
  "action_detail": "..."
}

When synthesizing the final answer, output:

jsonCopy
{
  "answer": "...",
  "citations": [{"claim": "...", "source_id": "...", "quote": "..."}],
  "remaining_uncertainty": "...",
  "sufficiency_summary": "..."
}

CONSTRAINTS

[CONSTRAINTS]

EXAMPLES

[EXAMPLES]

To adapt this template, start by defining your domain-specific thresholds for each rubric dimension. A medical application will have stricter authority and temporal requirements than a general knowledge assistant. Wire the tool definitions to your actual retrieval APIs, database queries, or clarification interfaces. Set MAX_ITERATIONS based on your latency budget—three to five iterations is typical for production RAG systems. The structured JSON output format is critical for programmatic control of the agent loop; your harness should parse the next_action field to route to the correct handler. For high-risk domains, add a human review step before the final synthesis is surfaced to users.

The Evidence Sufficiency Agent Prompt is designed to operate inside a tool-using agent loop, not as a single-turn classifier. The agent receives a user query and a set of retrieved passages, then iterates through a decision cycle: assess sufficiency, request additional retrieval or clarification if needed, and only proceed to answer generation when a defined threshold is met. The implementation harness must enforce this cycle, manage tool calls, validate the agent's structured output at each step, and prevent infinite loops or premature answers.

Wire the prompt as the system instruction for an agent that has access to a search_knowledge_base tool and a request_user_clarification tool. The agent's output at each turn must conform to a strict JSON schema with a decision field (ANSWER, RETRIEVE, CLARIFY, REFUSE) and a rationale field. The harness should parse this output and route accordingly: if RETRIEVE, execute the search tool with the agent's generated query, append results to the evidence set, and re-invoke the agent; if CLARIFY, surface the clarification question to the user and pause; if ANSWER, pass the accumulated evidence to a separate answer-generation prompt; if REFUSE, return the agent's refusal message. Implement a hard stop after a configurable maximum number of retrieval iterations (start with 3) to prevent runaway loops. Log every decision, the evidence set size, and the rationale for audit and debugging.

Validation is critical at each turn. Before routing on the decision field, confirm the JSON parses correctly and that the decision value is one of the allowed enum members. If the agent outputs ANSWER but the evidence set is empty or the sufficiency score in its own rationale is below a configured threshold, override with a REFUSE fallback and log the inconsistency. For high-stakes domains, insert a human-review step when the agent's confidence score falls in a middle band—auto-answer only above 0.9, refuse below 0.5, and queue for review between 0.5 and 0.9. Model choice matters: use a model with strong instruction-following and tool-use capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller models that may ignore the stopping criteria or hallucinate tool calls. Pair this agent prompt with an eval harness that measures end-to-end answer accuracy, refusal appropriateness, and retrieval iteration count against a golden dataset of queries with known sufficient and insufficient evidence states.