Vague Query Clarification Prompt for RAG Systems

This prompt is designed for a single, high-stakes job: intercepting an underspecified user query before it hits your retrieval engine and transforming it into a precise, disambiguated question. The ideal user is an AI engineer or product developer building a retrieval-augmented generation (RAG) system where broad, vague queries like "tell me about the policy" or "what's the update on the project?" produce noisy, low-quality retrieval results that degrade the final answer. The required context is a user's raw query string and, optionally, a list of available document scopes, timeframes, or entity types the system can search over. The prompt's output is not an answer; it is a single, targeted clarification question designed to narrow the search space before any retrieval call is made.

You should not use this prompt when the user's query is already specific enough to produce high-quality retrieval results, or when the cost of a clarification turn is higher than the cost of a broad search. For example, in a latency-sensitive customer-facing chatbot where users expect an immediate answer, adding a clarification step can create unacceptable friction. This prompt is also inappropriate for systems that lack a mechanism to pause the workflow, present a question to the user, and resume with the clarified input. If your architecture cannot hold state between turns, a clarification prompt will break the user experience. In these cases, consider a query rewriting or expansion prompt that improves retrieval quality without user interaction.

Before implementing this prompt, define your evaluation criteria. You should measure two things: the improvement in retrieval precision after clarification, and the user friction tolerance rate—how often users abandon the interaction when asked for clarification. A successful deployment will show a measurable lift in answer accuracy without a significant drop in task completion rates. Start by logging every instance where the prompt fires, the original query, the generated clarification question, and the user's response. Use these logs to tune the prompt's threshold for ambiguity and to identify queries that should have been clarified but weren't. In high-compliance domains like legal or healthcare, always log the clarification step as part of the audit trail to demonstrate that the system did not silently guess the user's intent.

This prompt template is designed to sit between the user's input and your retrieval step. Its job is to evaluate whether a query is specific enough to produce high-quality, relevant chunks from your vector store or search index. When the query is too vague, the prompt generates a single, precise clarification question that narrows the scope—such as asking for a time range, a specific product name, a document type, or a comparison dimension. The output is structured so your application can branch: if needs_clarification is true, surface the question to the user; if false, pass the original or rewritten query directly to retrieval. This prevents the expensive and user-facing failure mode of returning a broad, low-signal set of documents that waste context window and erode trust.

textCopy
You are a query clarity classifier for a retrieval-augmented generation (RAG) system. Your task is to evaluate whether a user's query is specific enough to retrieve relevant, targeted documents from a knowledge base. If the query is too vague, you must generate a single, precise clarification question that narrows the scope before retrieval proceeds.

## INPUT
User Query: [USER_QUERY]
Available Knowledge Domains: [DOMAINS]
Conversation History (last 3 turns): [HISTORY]

## CLASSIFICATION RULES
A query is VAGUE if it has any of these properties:
- Missing temporal scope (e.g., "recent," "latest" without a defined window)
- Missing entity or subject specification (e.g., "the report," "that issue" without a clear referent)
- Overly broad category (e.g., "security stuff," "performance problems")
- Ambiguous comparison target (e.g., "better" without stating better than what)
- Multiple valid interpretations across different domains in [DOMAINS]

A query is SPECIFIC if it contains enough detail for a targeted retrieval query with clear scope boundaries.

## OUTPUT SCHEMA
Return ONLY valid JSON with this exact structure:
{
  "needs_clarification": boolean,
  "vagueness_type": string or null,
  "clarification_question": string or null,
  "suggested_retrieval_query": string or null
}

## FIELD RULES
- `needs_clarification`: true if the query is vague, false if specific enough to retrieve.
- `vagueness_type`: If vague, classify as one of ["missing_temporal_scope", "missing_entity", "overly_broad", "ambiguous_comparison", "multi_domain_ambiguity"]. Use null if specific.
- `clarification_question`: If vague, write ONE clear, concise question that asks for the missing information. Do not ask multiple questions. Do not suggest answers. Make it easy for the user to answer in a few words. Use null if specific.
- `suggested_retrieval_query`: If specific, provide an optimized retrieval query. If vague, provide the best-effort query that would be used if the user refuses to clarify (for logging purposes only).

## CONSTRAINTS
- Do NOT answer the user's query. Only classify and optionally generate a clarification question.
- Do NOT generate a clarification question when the query is already specific.
- Do NOT ask the user to repeat information already present in [HISTORY].
- If the query is vague but [HISTORY] contains the missing information, treat the query as specific and use the history to fill in the retrieval query.
- For multi_domain_ambiguity, the clarification question must ask the user to choose which domain they mean.

## EXAMPLES

Example 1:
User Query: "Tell me about the outage"
Domains: ["incident_reports", "status_page_history", "postmortems"]
History: []
Output:
{
  "needs_clarification": true,
  "vagueness_type": "missing_entity",
  "clarification_question": "Which outage are you referring to? Please provide the incident ID, affected service name, or date it occurred.",
  "suggested_retrieval_query": "outage incident report"
}

Example 2:
User Query: "How do I configure the widget?"
Domains: ["product_docs", "api_reference", "troubleshooting_guides"]
History: [{"user": "I'm working with the dashboard v2 API", "assistant": "I can help with the dashboard v2 API. What do you need?"}]
Output:
{
  "needs_clarification": false,
  "vagueness_type": null,
  "clarification_question": null,
  "suggested_retrieval_query": "dashboard v2 API widget configuration"
}

Example 3:
User Query: "What's the best option?"
Domains: ["pricing_plans", "deployment_models", "support_tiers"]
History: [{"user": "I need to choose a plan for my team of 50", "assistant": "We have several plans. Are you interested in pricing, deployment, or support?"}]
Output:
{
  "needs_clarification": true,
  "vagueness_type": "multi_domain_ambiguity",
  "clarification_question": "Are you asking about the best pricing plan, deployment model, or support tier for your team of 50?",
  "suggested_retrieval_query": "best plan for 50 person team"
}

## RISK LEVEL
[RISK_LEVEL]

## CURRENT QUERY
User Query: [USER_QUERY]
Domains: [DOMAINS]
History: [HISTORY]

To adapt this template for your system, start by populating [DOMAINS] with the actual knowledge domains, collections, or indexes available in your RAG pipeline. This list is critical—it constrains the model's ambiguity detection to real retrieval targets rather than abstract categories. The [HISTORY] placeholder should receive the last 2–4 conversation turns in a structured format; include enough context to resolve anaphora and implicit references but not so much that the model loses the signal of the current turn. The [RISK_LEVEL] field is a control lever: set it to "low" for exploratory search where broad results are acceptable, "medium" for customer-facing support where a bad retrieval wastes time, or "high" for regulated domains where incorrect information has compliance consequences. At higher risk levels, you should add an additional constraint to the prompt requiring the model to err toward asking for clarification rather than proceeding with an underspecified query. If your system has a maximum question length for UI display, add a [MAX_QUESTION_LENGTH] constraint and validate the output before rendering it to the user.

This prompt is designed to sit as a pre-retrieval gate in your RAG pipeline. Before any vector search or knowledge base query is executed, the user's original query is passed to this prompt. The model's output is then parsed to determine whether to proceed with retrieval or to interrupt the flow and return a clarification question to the user. The harness must enforce a strict contract: if the action field is clarify, the system must not perform a search. Instead, it must surface the clarification_question to the user and wait for a refined query. This prevents the system from executing a broad, low-quality search that wastes resources and produces a poor answer.

To integrate this, wrap the prompt in a lightweight application function. First, call the LLM with the prompt template and the user's [USER_QUERY]. Parse the JSON response and validate the schema: action must be either proceed or clarify. If clarify, the clarification_question string must be non-empty. If validation fails, implement a retry strategy with a maximum of two attempts, feeding the raw output and a schema error message back to the model. If validation still fails after retries, log the failure and fall back to a safe default—either escalating to a human reviewer or proceeding with a conservative keyword search, depending on your application's risk tolerance. All clarification events should be logged with the original query, the generated question, and the user's eventual refined query to measure the prompt's effectiveness over time.

Model choice matters here. A smaller, faster model (like GPT-4o-mini or Claude Haiku) is ideal for this gating task to minimize latency, as it is a simple classification and generation step. However, you must evaluate the model's ability to correctly identify truly ambiguous queries without over-clarifying on simple, well-defined requests. Implement an eval harness that runs a golden dataset of ambiguous and unambiguous queries through the prompt. Measure the Clarification Rate (how often it asks for clarification) against your target user friction tolerance. A high false-positive rate will annoy users, while a high false-negative rate will lead to poor retrieval quality. Monitor both metrics in production and adjust the prompt's [CONSTRAINTS] or [EXAMPLES] to tune this trade-off.