Clarifying Question Generation Prompt Template

This prompt is for conversational search and RAG assistant developers who need to resolve underspecified user queries before retrieval. When a user asks a question that lacks a critical constraint—such as a time range, a document type, a specific entity, or a geographic scope—the model generates a single clarifying question that targets the missing information. The output includes both the question text and a structured schema for capturing the user's answer, so the application layer can inject the resolved constraint back into the retrieval pipeline. Use this prompt when the cost of retrieving irrelevant results is higher than the cost of one clarification turn. This is common in enterprise knowledge bases where a query like 'show me the policy' could return HR, IT, or legal documents with vastly different implications, or in customer-facing assistants where 'what's the status of my order' requires an order ID to avoid exposing another customer's data.

The prompt is designed for a specific architectural pattern: a pre-retrieval clarification gate. Wire it into your application so that the retrieval pipeline is blocked until either the query is deemed fully specified or the user's answer to the clarifying question is received and validated. The harness should enforce a maximum of one clarification turn before falling back to a broad retrieval or escalating to a human agent. Do not use this prompt when the query is already fully specified, when the user has explicitly requested a fast answer without interruption, or when the missing constraint would not materially change retrieval results. For example, a query like 'what is the capital of France' requires no clarification, while 'what's the weather' might need location but the cost of a wrong default is low enough to skip the turn. A common failure mode is over-clarification: asking for a date range on a query like 'company holiday schedule' when the knowledge base contains only one relevant document. Test for this by measuring the clarification rate against a golden set of queries where human annotators have marked whether clarification is warranted.

Before deploying, define the set of constraint types your system cares about—time, entity, document type, geography, authority level—and ensure your eval set covers each. Log every clarification event with the original query, the generated question, the user's response, and the final retrieval results. This trace is essential for tuning the prompt's threshold for intervention. If you operate in a regulated domain, add a human review step for clarification questions generated on queries tagged as high-risk, and never use the user's answer to directly mutate a database or trigger an action without confirmation. The next section provides the copy-ready prompt template you can adapt to your domain's specific constraint taxonomy.

This prompt template instructs the model to analyze an underspecified user query and generate a single, concise clarifying question that targets the most critical missing constraint. The output includes both the question text and a structured schema for capturing the user's response, making it directly usable in a conversational AI or RAG application. Replace each square-bracket placeholder with your specific inputs before sending the prompt to the model.

textCopy
You are a query disambiguation assistant for a retrieval system. Your job is to detect when a user query is underspecified and generate a single clarifying question that resolves the most important missing constraint.

## INPUT
User Query: [USER_QUERY]

## CONTEXT
Conversation History: [CONVERSATION_HISTORY]
Domain: [DOMAIN_DESCRIPTION]
Knowledge Base Summary: [KNOWLEDGE_BASE_SUMMARY]

## CONSTRAINTS
- Generate exactly one clarifying question.
- The question must target the constraint that, if resolved, would most narrow the retrieval space.
- Do not ask questions that are already answered in the conversation history.
- The question must be neutral and not lead the user toward a specific answer.
- Use the user's own terminology where possible.
- If the query is already sufficiently specific, output "NO_CLARIFICATION_NEEDED" as the question.

## OUTPUT_SCHEMA
Return a JSON object with the following structure:
{
  "needs_clarification": boolean,
  "missing_constraint_type": string (one of: "entity", "time_range", "location", "document_type", "scope", "authority_level", "comparison_target", "other"),
  "clarifying_question": string,
  "response_schema": {
    "type": string (one of: "single_choice", "multiple_choice", "free_text", "date_range", "entity_selection"),
    "options": array of strings (only for single_choice and multiple_choice),
    "validation": string (regex or constraint description for free_text and date_range)
  },
  "rationale": string (brief explanation of why this constraint matters for retrieval)
}

## EXAMPLES
User Query: "What's the revenue?"
Conversation History: []
Domain: Enterprise financial reporting
Output:
{
  "needs_clarification": true,
  "missing_constraint_type": "entity",
  "clarifying_question": "Which company or business unit's revenue are you asking about?",
  "response_schema": {
    "type": "entity_selection",
    "options": [],
    "validation": "non-empty string matching a known entity"
  },
  "rationale": "Revenue queries require a specific entity to retrieve the correct financial report."
}

User Query: "Show me the latest incident report."
Conversation History: [{"role": "user", "content": "I'm looking at the US East region."}]
Domain: Cloud operations
Output:
{
  "needs_clarification": true,
  "missing_constraint_type": "time_range",
  "clarifying_question": "What time period should the incident report cover? For example, the last 24 hours, this week, or a specific date?",
  "response_schema": {
    "type": "date_range",
    "options": [],
    "validation": "ISO 8601 date or relative range expression"
  },
  "rationale": "The region is known from conversation history, but the time range is unspecified and would significantly change which incidents are retrieved."
}

To adapt this template, replace [USER_QUERY] with the raw user input, [CONVERSATION_HISTORY] with prior turns in the session, [DOMAIN_DESCRIPTION] with a short description of the application domain, and [KNOWLEDGE_BASE_SUMMARY] with a summary of the available document types, entities, and metadata fields. The response_schema field in the output is designed to be passed to your frontend to render the appropriate input control. Before deploying, test the prompt against a set of queries with known missing constraints and verify that the generated questions are relevant, non-leading, and correctly classified by missing_constraint_type. In high-stakes domains, route outputs where needs_clarification is true but clarifying_question is empty or nonsensical to a human reviewer before presenting the question to the user.

This prompt is designed to sit between the user's initial query and the retrieval step in a conversational search or RAG pipeline. When the system detects an underspecified or ambiguous query, it calls this prompt to generate a clarifying question. The output is not a final answer but a structured payload containing the question text and a response schema. The application layer is responsible for presenting the question to the user, capturing the response, and merging the clarified constraint back into the original query before proceeding with retrieval. Do not send the clarifying question directly to the retrieval index—it is a user-facing artifact, not a search query.

The implementation harness should enforce a strict schema on the model's output. Expect a JSON object with at minimum clarifying_question (string), target_constraint (string describing what is missing), and response_schema (an object defining the expected user reply format, such as {"type": "single_choice", "options": ["Q1 2024", "Q2 2024", "Q3 2024"]} or {"type": "free_text", "expected_format": "date range"}). Validate this schema immediately after generation. If the model returns malformed JSON, missing required fields, or a clarifying_question that is empty or identical to the input query, discard the output and retry with a stricter instruction appended to the prompt, such as 'You MUST return valid JSON with all required fields.' After two consecutive validation failures, fall back to a canned clarifying question like 'Could you provide more specific details about your request?' and log the failure for prompt debugging. For high-stakes domains such as healthcare or finance, route the generated question to a human reviewer before showing it to the user. The review check should confirm the question is non-leading, relevant to the detected ambiguity, and does not introduce new assumptions.

Model choice matters here. This prompt requires strong instruction-following and structured output discipline. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are all suitable. Avoid smaller or older models that struggle with simultaneous JSON schema adherence and conversational tone. Set temperature to 0.2 or lower to reduce variance in question phrasing while preserving natural language quality. If your application uses a multi-turn session, include the last N conversation turns in the [CONTEXT] placeholder so the model can avoid asking questions the user already answered. Log every generated clarifying question along with the detected ambiguity type, the user's response, and whether the clarified query improved downstream retrieval metrics. This trace data is essential for tuning the ambiguity detection threshold and identifying patterns where the clarifying question itself caused user confusion or drop-off.