Clarify-vs-Route Decision Prompt Template

This prompt is a binary decision gate for interactive triage flows. Its job is to decide whether the system should interrupt the user with a clarifying question or proceed to route the request using the information already available. It is designed for system architects and platform engineers who need a deterministic, auditable decision point when a classification confidence score falls into a gray zone—where the cost of a wrong route (e.g., sending a billing dispute to a general FAQ queue) exceeds the cost of a brief user interruption. The prompt does not perform initial classification, generate the clarification question, or execute the final route; it only makes the 'ask vs. act' decision based on pre-computed inputs.

You should use this prompt when you have already classified the user's input and scored its confidence, but that confidence sits between your 'auto-route' and 'auto-reject' thresholds. The prompt requires a structured input containing the top classification label, its confidence score, the cost of a misroute, the cost of asking for clarification, and the downstream error impact. It is not suitable for initial intent detection, for generating the clarification question itself, or for making routing decisions when confidence is very high or very low—those cases should be handled by deterministic threshold rules in your application code. Pair this prompt with a Classification Confidence Scoring Prompt and a Clarification Question Generation Prompt to build a complete triage loop.

Before deploying, define your cost parameters rigorously. The 'misroute cost' should reflect the real operational impact (e.g., time wasted, customer escalation risk, SLA breach), not an abstract number. The 'clarification cost' should account for user friction, session abandonment risk, and latency. If these costs are not calibrated against your production data, the prompt will default to always asking or always routing, defeating its purpose. Implement a human review sampling step for decisions made near the boundary, and log every decision with its inputs for audit and threshold tuning. Do not use this prompt in asynchronous or batch processing pipelines where user interruption is impossible; in those cases, route to a fallback queue instead.

This prompt template implements the core clarify-vs-route decision logic. It forces the model to choose between two actions—asking a targeted clarifying question or proceeding with routing—based on a structured assessment of ambiguity cost, clarification cost, and downstream error impact. The template is designed to be called at the decision boundary after classification confidence has been scored but before the input is dispatched to a handler, queue, or human reviewer.

codeCopy
SYSTEM: You are a triage decision engine. Your only job is to decide whether an ambiguous or low-confidence input should be clarified with the user or routed with the available information. You must choose exactly one action and justify it with the cost analysis provided.

INPUT:
- User request: [USER_INPUT]
- Classification result: [CLASSIFICATION_LABEL]
- Confidence score: [CONFIDENCE_SCORE] (0.0 to 1.0)
- Top alternative classifications: [TOP_K_ALTERNATIVES]
- Ambiguity severity: [AMBIGUITY_SEVERITY] (none | low | medium | high | critical)
- Ambiguous spans: [AMBIGUOUS_SPANS]

COST PARAMETERS:
- Cost of wrong routing: [WRONG_ROUTE_COST] (low | medium | high | critical)
- Cost of asking user: [CLARIFICATION_COST] (low | medium | high)
- User tolerance for clarification: [USER_TOLERANCE] (low | medium | high)
- Available clarification budget: [CLARIFICATION_BUDGET] (remaining questions allowed)

CONSTRAINTS:
- If confidence >= [HIGH_CONFIDENCE_THRESHOLD], route immediately without clarification.
- If confidence < [LOW_CONFIDENCE_THRESHOLD] and wrong_route_cost is high or critical, clarify.
- If clarification_budget is 0, route with best available classification and flag uncertainty.
- If ambiguity_severity is critical and wrong_route_cost is critical, escalate to human; do not clarify or route.
- If multiple spans are ambiguous and they point to different intents, clarify before routing.

OUTPUT_SCHEMA:
{
  "decision": "clarify" | "route" | "escalate",
  "confidence": 0.0,
  "reasoning": "string explaining the cost trade-off that drove the decision",
  "clarification_question": "string | null (required if decision is clarify, must target the specific ambiguous span)",
  "route_target": "string | null (required if decision is route, the queue, handler, or model to route to)",
  "fallback_action": "string | null (required if decision is escalate, the escalation path)",
  "uncertainty_flags": ["string"]
}

EXAMPLES:
[FEW_SHOT_EXAMPLES]

Now decide: clarify, route, or escalate.

Adapt this template by replacing each square-bracket placeholder with runtime values from your classification pipeline. The cost parameters—wrong_route_cost, clarification_cost, and user_tolerance—should be configured per use case and ideally sourced from a policy configuration rather than hardcoded. The few-shot examples should cover at least three scenarios: a clear route decision, a clear clarify decision, and a boundary case where escalation is correct. Test the template against a golden set of decisions before deployment, and log every decision with its reasoning for audit and threshold calibration.

The Clarify-vs-Route prompt is a binary decision point in a larger triage pipeline. It should be called only after upstream classification has returned a label and a confidence score, and only when that confidence falls below your system's auto-route threshold. The prompt's job is narrow: given the original input, the top classification candidate, its confidence, and the cost context, decide whether to ask a clarifying question or route with the available information. Do not use this prompt as a standalone classifier or as a replacement for upstream intent detection. It is a guardrail, not a router.

Wire the prompt into your application as a synchronous decision step with a strict timeout (500ms–2s depending on model and latency budget). The input payload must include: [USER_INPUT], [TOP_INTENT], [CONFIDENCE_SCORE], [ALTERNATIVE_INTENTS] (top 3 with scores), [CLARIFICATION_COST] (a numeric estimate of user friction, e.g., 1–10), [MISROUTE_COST] (a numeric estimate of downstream error impact, e.g., 1–10), and [CONTEXT] (any relevant conversation history or user profile signals). The output must conform to a strict JSON schema: { "decision": "clarify" | "route", "reasoning": "string", "clarification_question": "string | null", "routing_target": "string | null" }. Validate the output before acting on it. If decision is clarify but clarification_question is null or empty, treat it as a routing decision with a warning log. If decision is route but routing_target is null, fall back to the top intent from the upstream classifier. Use a JSON schema validator in your application layer—do not rely on the model to always produce valid JSON, even with strong format instructions.

Implement a retry policy with a maximum of 2 retries on validation failure. On the first failure, append the validation error message to the prompt and retry. On the second failure, log the full input and partial output, then default to a safe fallback: if [MISROUTE_COST] is greater than [CLARIFICATION_COST], clarify; otherwise, route to the top intent. This fallback logic should be hardcoded in your application, not left to the model. Log every decision with the full input payload, output payload, model used, latency, and retry count. These logs are essential for evaluating decision optimality over time and for tuning the [CLARIFICATION_COST] and [MISROUTE_COST] parameters against real user outcomes.

For model selection, use a fast, instruction-following model (GPT-4o-mini, Claude 3.5 Haiku, or equivalent) rather than a large reasoning model. This prompt requires structured decision-making, not deep reasoning, and latency matters in interactive triage flows. If you are operating in a regulated domain (healthcare, finance, legal), add a human review step when both [CLARIFICATION_COST] and [MISROUTE_COST] are above a defined threshold (e.g., both ≥7). The review step should present the original input, the model's decision, and the reasoning to a human operator before the clarification question is shown to the user or the routing decision is executed. Do not use this prompt to make decisions about safety-critical content without human-in-the-loop approval.

To evaluate the prompt's performance, build a test harness with 50–100 labeled examples covering clear-routing cases, clear-clarification cases, and ambiguous boundary cases. For each example, define the expected decision and acceptable clarification questions. Run the prompt against this test set on every change and measure decision accuracy, clarification question relevance (using a separate LLM judge), and schema compliance rate. Track these metrics over time and set alert thresholds: if decision accuracy drops below 90% or schema compliance drops below 99%, block the deployment and investigate. The most common failure mode is the model choosing to clarify when routing would have been sufficient, which increases user friction without improving outcomes. Monitor the ratio of clarify-to-route decisions in production and investigate spikes.