Clarification Question Generation Prompt for Ambiguous Queries

This prompt is for RAG system builders who need to resolve user intent before committing to an answer. Its job is to generate targeted clarification questions when a user query is ambiguous, underspecified, or could map to multiple distinct evidence paths. The ideal user is an AI engineer or product developer integrating a clarification step into a customer-facing Q&A system, where guessing the user's intent leads to irrelevant answers, wasted retrieval, and eroded trust. Use this prompt when the cost of a wrong answer is higher than the cost of asking a follow-up question—common in technical support, legal research, clinical information retrieval, and complex product documentation.

Do not use this prompt when queries are straightforward, when latency constraints forbid a second turn, or when the system has sufficient context (user history, account data, session state) to disambiguate without asking. It is also a poor fit for exploratory search where users expect broad results rather than a single precise answer. The prompt works best as a gate before the main synthesis step: if the clarification model returns questions, surface them to the user; if it returns an empty set, proceed to evidence retrieval and answer generation. This prevents the system from silently fabricating assumptions about what the user meant.

Before deploying, define what counts as 'ambiguous enough to ask.' Wire the prompt into a decision harness that measures unnecessary clarification rate (asking when the query was clear) and missed ambiguity rate (failing to ask when the query was genuinely underspecified). Log every clarification decision with the original query, generated questions, and whether the user's response actually changed the retrieval path. This trace data is essential for tuning the ambiguity threshold and defending the UX cost of the extra turn. Start with a high bar for asking—only when multiple reasonable interpretations would produce materially different answers—and lower it only if user feedback or answer quality metrics justify the interruption.

This template is designed to be dropped into a RAG pipeline immediately after retrieval and evidence sufficiency assessment. When the system determines that the available context cannot support a faithful answer—either because the query is underspecified, the retrieved documents are too broad, or multiple interpretations are possible—this prompt instructs the model to generate a small set of precise, non-leading clarification questions. The goal is to resolve ambiguity without hallucinating assumptions about user intent, and without asking questions that the retrieved context already answers.

textCopy
You are a clarification specialist in a retrieval-augmented Q&A system. Your job is to identify what is ambiguous in a user's query and ask targeted questions that will resolve that ambiguity before answer generation.

## INPUT
User Query: [USER_QUERY]
Retrieved Context (may be insufficient or ambiguous): [RETRIEVED_CONTEXT]

## CONSTRAINTS
- Do not answer the query. Only generate clarification questions.
- Do not ask questions that the retrieved context already answers.
- Do not ask leading questions that assume facts not in evidence.
- Generate between [MIN_QUESTIONS] and [MAX_QUESTIONS] questions.
- Each question must address a specific ambiguity in the query.
- If the query is clear and the context is sufficient, output an empty list.
- If the query is completely out of scope for the system, output a single refusal note instead of questions.

## OUTPUT_SCHEMA
Return a JSON object with the following structure:
{
  "needs_clarification": boolean,
  "ambiguity_type": "underspecified" | "multiple_interpretations" | "temporal_ambiguity" | "entity_ambiguity" | "scope_ambiguity" | "none",
  "questions": [
    {
      "question": "string",
      "targets": "what specific ambiguity this resolves",
      "options": ["option_a", "option_b"] or null
    }
  ],
  "refusal_note": null or "string explaining why clarification is impossible"
}

## EXAMPLES
Query: "What's the policy?"
Context: [Multiple policy documents from different departments]
Output: {"needs_clarification": true, "ambiguity_type": "entity_ambiguity", "questions": [{"question": "Which department's policy are you asking about?", "targets": "department scope", "options": null}]}

Query: "How do I reset it?"
Context: [Documentation covering password reset, factory reset, and network reset]
Output: {"needs_clarification": true, "ambiguity_type": "multiple_interpretations", "questions": [{"question": "What are you trying to reset—your password, the device to factory settings, or the network configuration?", "targets": "reset target", "options": ["password", "factory settings", "network configuration"]}]}

Adapt this template by adjusting the ambiguity_type enum to match your domain's common failure modes. For customer support systems, add types like product_ambiguity or entitlement_ambiguity. For legal RAG, add jurisdiction_ambiguity. The options field in each question is optional but valuable when you can present the user with a constrained choice rather than an open-ended text field—this reduces follow-up ambiguity and improves the downstream answer quality. If your system supports multi-turn conversation, wire the user's response to a clarification question back into the retrieval step before re-running synthesis. Always validate the output against the schema before presenting questions to the user; a malformed clarification question is worse than a generic fallback.

The clarification question prompt should sit between retrieval and answer synthesis in your RAG pipeline. When the initial retrieval returns low-confidence results, conflicting evidence, or the user query contains ambiguous entities, route the query to this prompt instead of proceeding directly to answer generation. The prompt receives the original user query, any retrieved context snippets, and a set of ambiguity categories you want the system to detect. It returns a structured clarification question or a signal that no clarification is needed. This routing decision should be logged with the query, retrieval scores, and clarification output for later evaluation.

Wire the prompt into an application harness that validates the output schema before showing anything to the user. The expected output is a JSON object with fields: needs_clarification (boolean), clarification_question (string or null), ambiguity_type (enum of your defined categories), and assumed_interpretations (array of strings describing what the model thinks the user might mean). Validate that needs_clarification is a strict boolean, that clarification_question is non-empty when needs_clarification is true, and that ambiguity_type matches one of your predefined categories. If validation fails, retry with the same prompt and append the validation error to the model's context as a correction instruction. After three failed retries, log the failure and escalate to a human review queue rather than guessing.

Model choice matters here. Use a model with strong instruction-following and JSON mode support, such as GPT-4o, Claude 3.5 Sonnet, or a fine-tuned variant if you have labeled clarification data. Avoid smaller or older models that tend to hallucinate clarification questions when the query is actually unambiguous. Set temperature low (0.0–0.2) to reduce variance in the boolean decision. If you are using a RAG architecture with multiple retrieval sources, pass the top-k passages from each source into the [CONTEXT] placeholder along with their retrieval scores so the model can assess whether ambiguity stems from conflicting evidence or missing information. Log every clarification decision with the query, retrieval scores, model response, and whether the user accepted or rejected the clarification. This log becomes your evaluation dataset for tuning the unnecessary clarification rate and missed ambiguity rate.

For high-stakes domains such as healthcare or legal, do not let the clarification question reach the user without human review if the ambiguity_type involves safety-critical distinctions. Build an approval step into the harness that routes certain ambiguity_type values to a review queue before the question is sent. Additionally, track the rate at which users ignore or reject clarification questions. A high rejection rate signals that the prompt is asking unnecessary questions, which degrades user trust. Use this metric to adjust the ambiguity detection thresholds or to add few-shot examples that teach the model when to proceed without clarification. The harness should also enforce a maximum of one clarification round per user turn to prevent infinite clarification loops.