Multi-Turn Abstention Decision Prompt Template

This prompt is for high-stakes RAG systems that operate across multiple conversation turns and must decide whether to answer, ask for clarification, or refuse. Unlike single-turn abstention prompts that evaluate only the current query and retrieved evidence, this template considers accumulated uncertainty from prior turns, failed clarification attempts, contradictory user signals, and escalating risk. Use this when wrong answers carry compliance, safety, or clinical consequences and the system needs a structured decision framework that spans the full conversation session.

This prompt belongs inside a guardrail layer that runs before answer generation, not as a post-hoc filter. It assumes you already have conversation history, retrieved evidence per turn, and a defined risk taxonomy. The template expects inputs including the full dialogue history, per-turn evidence sets, a risk classification schema, and a clarification attempt counter. It produces a structured abstention decision with one of three outcomes: answer, clarify, or refuse. Each outcome includes a confidence score, reasoning trace, and escalation flag. The prompt is designed to be wired into a pre-generation validation step where the decision gates whether the answer-generation prompt runs at all.

Do not use this prompt for low-stakes Q&A, single-turn retrieval, or systems where a wrong answer has no meaningful consequence. It adds latency and token cost that are only justified when the cost of a bad answer exceeds the cost of refusing to answer. Do not use it as a standalone safety layer without complementary retrieval quality checks, evidence grounding prompts, and human review escalation paths. The prompt is most effective when paired with a session-level state tracker that maintains uncertainty scores, clarification counts, and risk escalation history across turns. Before deploying, validate the abstention decision against a golden dataset of multi-turn conversations with known correct abstention points, and monitor production for over-refusal rates that degrade user trust.

This template is the core decision engine for high-stakes RAG systems that must track uncertainty across a conversation. Unlike a single-turn abstention prompt, this template requires the model to consider the history of prior clarifications, the user's patience, and whether continued ambiguity has crossed a risk threshold. The output is a structured decision—answer, clarify, or abstain—with a rationale that references specific prior turns and evidence gaps. Copy the template below and replace each square-bracket placeholder with your application's data before sending it to the model.

textCopy
You are a cautious question-answering system with access to retrieved evidence. Your primary directive is to avoid providing incorrect or unsupported information, especially when the stakes are high. You must decide whether to answer the current question, ask a clarifying question, or abstain entirely.

## CONVERSATION HISTORY
[CONVERSATION_HISTORY]
<!-- Format: A list of prior turns, each with role, message, and any system actions (e.g., clarification requests, abstentions). Example:
- user: "What is the capital of France?"
- assistant: "Paris."
- user: "What is its population?"
- assistant: [ABSTAIN] "I cannot find the population of Paris in the provided documents."
-->

## CURRENT USER QUESTION
[CURRENT_QUESTION]

## RETRIEVED EVIDENCE
[RETRIEVED_EVIDENCE]
<!-- Format: A list of passages with source identifiers. Example:
- [doc_1] "Paris is the capital of France."
- [doc_2] "The city's population was 2.1 million in 2023."
-->

## ABSTENTION POLICY
[ABSTENTION_POLICY]
<!-- A description of when to abstain, clarify, or answer. Example:
- Abstain if the evidence does not support a factual answer.
- Clarify if the question is ambiguous but answerable with more detail.
- Answer if the evidence is sufficient and directly addresses the question.
- Escalate to abstention if the user has been asked to clarify the same point more than [MAX_CLARIFICATION_ATTEMPTS] times.
-->

## RISK LEVEL
[RISK_LEVEL]
<!-- One of: low, medium, high, critical. Higher risk levels should bias the system toward abstention and require stronger evidence for answering. -->

## OUTPUT SCHEMA
You must respond with a valid JSON object conforming to this schema:
{
  "decision": "answer" | "clarify" | "abstain",
  "rationale": "A concise explanation referencing specific evidence, prior turns, and policy rules.",
  "clarification_question": "If decision is 'clarify', a specific question to resolve the ambiguity. Otherwise null.",
  "answer": "If decision is 'answer', the grounded answer with citations. Otherwise null.",
  "uncertainty_indicators": ["A list of specific reasons for uncertainty, e.g., 'conflicting sources', 'stale evidence', 'ambiguous user intent'."]
}

## CONSTRAINTS
- Do not fabricate information. If the evidence is insufficient, you must abstain or clarify.
- If you have already asked for clarification on this point in [CONVERSATION_HISTORY] and the user has not resolved it, bias toward abstention.
- Consider the [RISK_LEVEL]: a 'critical' risk level means a wrong answer is unacceptable.

To adapt this template, start by defining your [ABSTENTION_POLICY] in concrete, testable terms. A vague policy like 'be careful' will produce inconsistent decisions. Instead, specify exact conditions: 'Abstain if no passage has a relevance score above 0.8' or 'Clarify if the question contains pronouns without a clear referent in the last two turns.' The [MAX_CLARIFICATION_ATTEMPTS] parameter inside the policy is critical for multi-turn behavior—set it to 1 or 2 to prevent infinite clarification loops that frustrate users. For the [CONVERSATION_HISTORY], include not just user and assistant messages but also metadata about prior abstention decisions and clarification requests. This gives the model the context it needs to recognize when a question has already been asked and failed. Finally, wire the [RISK_LEVEL] to a real application signal—for example, route questions from authenticated doctors in a clinical setting as 'high' risk, while anonymous FAQ queries might be 'low' risk. The next step is to build a validation harness that checks the JSON output against the schema and logs every abstention decision for offline review.

The Multi-Turn Abstention Decision prompt is not a standalone component; it is a decision node in a larger RAG pipeline. Wire it after retrieval and evidence ranking but before answer synthesis. The prompt receives the current user query, conversation history, retrieved evidence, and any prior abstention signals (e.g., previous low-confidence flags or unanswered clarification requests). Its output is a structured decision—ANSWER, CLARIFY, ABSTAIN, or ESCALATE—along with a confidence score and reasoning. This decision gates the next step: if ANSWER, proceed to synthesis; if CLARIFY, route to the clarification prompt; if ABSTAIN, return a canned refusal with caveats; if ESCALATE, queue for human review. Do not use this prompt for low-stakes chitchat or domains where a wrong answer carries trivial cost.

Integration pattern. Call this prompt as a synchronous step within your RAG orchestrator. Pass the following inputs: [CURRENT_QUERY], [CONVERSATION_HISTORY] (last N turns, with system answers and user feedback), [RETRIEVED_EVIDENCE] (top-K passages with source metadata), [PRIOR_ABSTENTION_STATE] (a running struct tracking previous CLARIFY or ABSTAIN decisions and whether they were resolved), and [RISK_THRESHOLD] (a numeric or categorical threshold set by policy, e.g., "high" for clinical or legal domains). Parse the model's JSON output with a strict schema validator. If the output fails schema validation, retry once with the error message injected into the prompt as [PREVIOUS_OUTPUT_ERROR]. If the retry also fails, default to ESCALATE and log the failure for review.

Human-in-the-loop. For regulated or high-risk domains, ESCALATE must trigger a human review queue, not a fallback answer. Store the full decision payload—including the model's reasoning, confidence, and the evidence it considered—in your audit log. The human reviewer should see the conversation context, the retrieved evidence, and the model's abstention rationale. After review, record the human decision (ANSWER, CLARIFY, ABSTAIN) and use it to update the session's [PRIOR_ABSTENTION_STATE]. This feedback loop prevents the system from repeatedly escalating the same pattern without learning. Consider using a simpler, faster model (e.g., a classifier fine-tuned on your escalation logs) as a pre-filter before calling the full multi-turn abstention prompt, to reduce latency and cost on obvious cases.

Eval and monitoring. Before shipping, build an eval harness that tests the abstention decision across conversation trajectories. Key scenarios: (1) a single low-confidence turn should trigger CLARIFY, not immediate ABSTAIN; (2) two consecutive CLARIFY attempts with no resolution should escalate to ABSTAIN or ESCALATE; (3) new evidence that resolves prior uncertainty should flip the decision back to ANSWER; (4) topic shifts should reset the abstention state. Log every decision with trace IDs that link the prompt version, inputs, output, and final routing action. Monitor the rate of ESCALATE decisions, schema validation failures, and human overrides. A sudden spike in escalations often indicates a retrieval pipeline regression or a prompt drift issue. Wire these metrics into your production dashboards alongside answer quality scores.