Answer Abstention with Evidence Explanation Prompt

This prompt is for AI product teams building cited answer systems where a wrong or unsupported answer is worse than no answer at all. The primary job-to-be-done is to produce a graceful, auditable abstention message when retrieved evidence is insufficient, conflicting, or irrelevant to the user's question. The ideal user is an engineering lead or developer integrating a RAG pipeline into a product where user trust and compliance depend on knowing when the system chose not to answer. Required context includes the user's original question, the full set of retrieved evidence passages, and any confidence thresholds or evidence sufficiency rules your application has already evaluated.

Do not use this prompt when the system should always attempt an answer regardless of evidence quality, such as in creative brainstorming or low-stakes chat. It is also the wrong tool when your application layer can deterministically decide to abstain based on retrieval scores alone—reserve this prompt for cases where the decision requires semantic reasoning about evidence relevance, contradiction, or subtle insufficiency. The prompt assumes you have already run retrieval and have a set of candidate passages; it does not initiate new retrieval or expand search windows. For those workflows, pair this with a retrieval expansion prompt from the same pillar.

Before wiring this into production, define your abstention policy clearly: what evidence quality thresholds trigger abstention, what tone the abstention message should use, and whether the user should see the evidence that was found. This prompt is high-stakes by nature—an incorrect abstention erodes user confidence, while a failure to abstain spreads hallucinated claims. Always pair it with an evaluation step that checks abstention quality, and consider human review for domains like healthcare, legal, or finance where an inappropriate abstention could cause harm.

This prompt template is designed for RAG systems and document-intelligence applications that must gracefully decline to answer when retrieved evidence is insufficient. Instead of generating a hallucinated response or a generic refusal, the model produces a structured abstention that explains its evidence-seeking process, summarizes what it found, and justifies why it cannot answer. This builds user trust and creates an auditable record for compliance workflows.

textCopy
You are an evidence-grounded answering system. Your primary obligation is to answer only when the provided evidence supports a complete and accurate response. When evidence is insufficient, you must abstain clearly and explain why.

## INPUT

**User Question:** [USER_QUESTION]

**Retrieved Evidence:**
[RETRIEVED_EVIDENCE]

**Evidence Retrieval Summary:**
- Sources searched: [SOURCE_LIST]
- Retrieval queries used: [QUERY_LIST]
- Retrieval timestamp: [TIMESTAMP]

## CONSTRAINTS

[CONSTRAINTS]

## OUTPUT SCHEMA

Respond in valid JSON matching this schema:

{
  "decision": "abstain" | "answer",
  "abstention": {
    "reason": "evidence_insufficient" | "evidence_conflicting" | "out_of_scope" | "retrieval_failure",
    "evidence_sought": "What specific information was needed to answer the question",
    "evidence_found": "Summary of what the retrieved evidence actually contained",
    "gap_explanation": "Why the found evidence is insufficient to answer the question",
    "suggested_next_steps": "What the user could try, such as rephrasing, providing documents, or contacting a subject-matter expert"
  },
  "answer": null,
  "evidence_citations": [],
  "confidence": {
    "score": 0.0,
    "rationale": "Why confidence is below the answering threshold"
  }
}

## INSTRUCTIONS

1. Review the user question and all retrieved evidence carefully.
2. Determine whether the evidence contains sufficient information to answer the question completely and accurately.
3. If evidence is sufficient, set decision to "answer" and populate the answer field with cited claims.
4. If evidence is insufficient, set decision to "abstain" and populate the abstention object with a thorough explanation.
5. Never fabricate evidence, citations, or claims not present in the provided evidence.
6. If evidence is conflicting, explain the conflict rather than choosing sides arbitrarily.
7. If the question is out of scope for the available evidence, state this clearly.
8. If retrieval itself failed or returned empty results, acknowledge this as a retrieval failure.

## EXAMPLES

[EXAMPLES]

## RISK LEVEL

[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with your application's specific values. The [CONSTRAINTS] block should include domain-specific rules such as regulatory requirements, tone guidelines, or evidence sufficiency thresholds. The [EXAMPLES] block is critical for teaching the model the desired abstention style—include at least two examples showing different abstention reasons. For high-risk domains like healthcare or legal, set [RISK_LEVEL] to "high" and add a mandatory human-review flag in the output schema. Always validate the JSON output against the schema before surfacing the abstention to users, and log every abstention decision with the evidence gap explanation for observability and audit trails.

The Answer Abstention with Evidence Explanation prompt is not a standalone component. It sits inside a broader RAG or document-intelligence harness that has already attempted retrieval, ranked passages, and generated a candidate answer. When the candidate answer's citation coverage, confidence scores, or evidence sufficiency checks fall below configured thresholds, the harness should route the request to this abstention prompt instead of returning a low-quality answer to the user. The prompt expects structured inputs: the original user query, the retrieved evidence set with source identifiers, the candidate answer that triggered the abstention path, and the specific failure reasons from upstream validators. Without these inputs, the prompt cannot produce a useful explanation of what was sought, what was found, and why it was insufficient.

Wire the prompt into your application as a recovery step within a citation validation pipeline. After your primary answer generation step, run a citation completeness check or source-to-claim alignment validator. If the validator returns coverage_score below your threshold (e.g., < 0.7) or flags unsupported claims, call this abstention prompt with the validator's failure details in the [FAILURE_REASONS] placeholder. Parse the output against the expected abstention schema: { abstention_message, evidence_sought, evidence_found, insufficiency_reason, suggested_next_steps }. Validate that insufficiency_reason is non-empty and maps to one of your defined insufficiency categories (e.g., no_relevant_sources, partial_coverage, conflicting_sources, low_authority). If the abstention output itself fails schema validation, retry once with the validation error injected into [CONSTRAINTS]. After two failed abstention attempts, escalate to a human review queue with the full trace attached. Log every abstention event with the original query hash, retrieval parameters, validator scores, and final abstention reason for observability and threshold tuning.

Model choice matters here. Use a model with strong instruction-following and low hallucination rates for abstention tasks—GPT-4o, Claude 3.5 Sonnet, or equivalent. Avoid smaller or older models that may fabricate evidence explanations or invent source details. Set temperature to 0 or near-zero to maximize output consistency. If your system supports tool use, consider exposing a log_abstention_event tool that the model can call to record structured metadata alongside the user-facing message. Do not use this prompt as a first-resort answer generator; it is a recovery path. The primary path should always attempt grounded answer generation first. Monitor abstention rates by insufficiency category in production—rising no_relevant_sources rates may indicate retrieval degradation, while rising low_authority rates may signal source corpus quality issues. Treat abstention as a signal, not just a safety valve.