Evidence Insufficiency Recovery Prompt

This prompt is a recovery gate for RAG pipelines and cited-answer systems. It activates after an initial answer is generated but before it reaches the user. Its job is to evaluate whether the retrieved evidence is sufficient to support the claims made in the answer. When evidence is too thin, ambiguous, or off-topic, the prompt forces one of two recovery paths: abstention with a clear explanation of the gap, or a signal to trigger expanded retrieval with reformulated queries. Use this prompt when user trust, compliance, or auditability depends on every claim being traceable to a source. Do not use this prompt as a primary answer generator; it is a quality-control step that sits between generation and delivery.

The ideal user is an AI engineer or platform developer building a production cited-answer system where hallucination risk carries real consequences—regulatory filings, clinical summaries, legal research, financial analysis, or customer-facing knowledge bases. Required context includes the original user query, the generated answer, the full set of retrieved evidence passages, and any retrieval metadata such as scores or source identifiers. Without this context, the prompt cannot perform meaningful sufficiency evaluation. The prompt expects evidence passages to be clearly delimited and indexed so it can reference specific sources in its assessment.

Do not deploy this prompt when the retrieval corpus is known to be incomplete or when the application can tolerate unsupported statements. It will increase latency and token consumption, and it will produce abstentions that may frustrate users who expect an answer regardless of evidence quality. Also avoid using this prompt as a substitute for proper retrieval quality—if your retriever consistently returns irrelevant passages, fix the retrieval pipeline first. This prompt is a safety net, not a replacement for upstream retrieval performance. Before wiring it into production, define your sufficiency threshold: how many supporting passages are required, what relevance score floor applies, and whether partial support is acceptable or triggers full abstention.

This template is the core instruction set for an Evidence Insufficiency Recovery step. It forces the model to evaluate whether the provided context can support a faithful answer before generating one. The prompt uses square-bracket placeholders for all dynamic inputs, making it safe to paste into a recovery harness that populates variables at runtime. The template is designed to be called after an initial retrieval step returns candidate documents but before the final answer is shown to the user.

textCopy
You are an evidence sufficiency evaluator. Your job is to determine whether the provided context contains enough information to answer the user's question accurately and completely.

## INPUTS
User Question: [USER_QUESTION]
Retrieved Context: [RETRIEVED_CONTEXT]
Sufficiency Threshold: [SUFFICIENCY_THRESHOLD]

## INSTRUCTIONS
1. Read the user question and the retrieved context carefully.
2. Identify every factual claim that would be required to answer the question fully.
3. For each required claim, check whether the retrieved context provides direct, unambiguous support.
4. Classify the evidence as one of:
   - SUFFICIENT: All required claims are directly supported by the context.
   - PARTIALLY_SUFFICIENT: Some claims are supported, but others are missing or ambiguous.
   - INSUFFICIENT: The context lacks support for most or all required claims.
5. If the classification is SUFFICIENT, output the answer with inline citations.
6. If the classification is PARTIALLY_SUFFICIENT or INSUFFICIENT, do NOT generate an answer. Instead, output an abstention notice that includes:
   - The classification.
   - Which specific claims could not be supported.
   - Suggested retrieval queries to fill the gaps.

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "classification": "SUFFICIENT" | "PARTIALLY_SUFFICIENT" | "INSUFFICIENT",
  "supported_claims": ["claim1", "claim2"],
  "unsupported_claims": ["claim3"],
  "answer": "Only present if classification is SUFFICIENT. Include [CITATION:source_id] markers.",
  "abstention_notice": "Only present if classification is not SUFFICIENT. Explain what was missing.",
  "suggested_retrieval_queries": ["query1", "query2"]
}

## CONSTRAINTS
- Never fabricate citations or claim support that does not exist in the context.
- If you are unsure about support for any claim, classify as PARTIALLY_SUFFICIENT.
- Do not include the answer field unless classification is SUFFICIENT.
- Do not include the abstention_notice field unless classification is not SUFFICIENT.

Adapt the template by adjusting the [SUFFICIENCY_THRESHOLD] variable to match your risk tolerance. A strict threshold forces abstention for any ambiguity, which is appropriate for compliance-sensitive domains. A looser threshold allows partial answers with caveats, which may be acceptable for exploratory research use cases. The output schema can be extended with additional fields such as confidence_score or review_required if your harness needs to route outputs to human reviewers based on classification results.

Before deploying this prompt into production, pair it with a validation step that checks the output JSON against the schema. If the model returns SUFFICIENT but omits citation markers, or returns INSUFFICIENT but includes an answer field, the harness should reject the output and retry. For high-stakes workflows, route all PARTIALLY_SUFFICIENT outputs to a human review queue rather than attempting automatic regeneration.

The Evidence Insufficiency Recovery Prompt is not a standalone fix. It must sit inside a harness that controls retrieval scope, enforces evidence thresholds, and prevents infinite retry loops. The harness receives a generated answer and its associated retrieval context, runs the prompt to evaluate sufficiency, and then routes the output to one of three paths: accept the answer as-is, trigger expanded retrieval and regeneration, or abstain with an explanation. This routing decision should be deterministic based on the prompt's structured output, not on free-text heuristics.

Implement the harness as a stateful pipeline with a configurable retry budget. Start by defining a SufficiencyThreshold object with fields for minimum passage count, minimum cumulative relevance score, and required coverage ratio for the answer's factual claims. After the model returns its evaluation, validate the output against a strict schema that includes sufficiency_verdict (SUFFICIENT | INSUFFICIENT | ABSTAIN), gap_description, and retrieval_queries for the expanded search. If the verdict is INSUFFICIENT and the retry budget remains, feed the generated queries into your retrieval system with a wider window or relaxed filters, then regenerate the answer and re-evaluate. Log every evaluation attempt with the verdict, gap description, and retrieval parameters for trace analysis. If the budget is exhausted or the verdict is ABSTAIN, surface the abstention message to the user with the gap description intact.

Model choice matters here. Use a model with strong instruction-following for structured evaluation output, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller models that may conflate sufficiency evaluation with answer generation. The prompt should be called as a separate evaluation step, not as a prefix to the answer generation prompt, to keep the evaluator's judgment independent of the generator's biases. Add a human review gate when the system abstains on high-stakes queries or when the retry budget is exhausted without reaching sufficiency. Never silently drop an insufficient answer—always surface the evidence gap to the user or to an internal review queue.