Ambiguity Flagging Prompt for Overlapping Intents

This prompt is designed for platform engineers and AI infrastructure teams who are building intent classification middleware. Its primary job is to handle a critical failure mode in production routers: inputs that plausibly match multiple intent categories. When a user message like 'I can't log in and I need to cancel my subscription' hits a standard single-label classifier, the system either picks one intent and silently drops the other, or it forces a low-confidence tie-break that degrades downstream workflows. This prompt replaces that brittle behavior with a structured disambiguation output that surfaces all candidate intents, provides evidence spans from the input text, and recommends whether the system should split the request, ask for clarification, or route to a human.

The ideal user is an engineering lead or senior developer responsible for a routing layer that dispatches requests to separate agent pipelines, tool chains, or support queues. You should use this prompt when your production traffic contains compound or overlapping requests, when misclassification carries a high cost (e.g., a cancellation request routed to a billing FAQ agent), or when you need an auditable record of why the system chose one intent over another. Do not use this prompt for simple, single-intent classification where latency and token cost are the primary constraints; a lightweight zero-shot classifier with a confidence threshold is more appropriate for that use case. This prompt adds overhead by design, and that overhead is only justified when the cost of ambiguity exceeds the cost of the extra inference.

Before deploying this prompt, you must have a defined intent taxonomy and a clear policy for how your system handles multi-intent inputs. The prompt will not invent your routing logic; it will expose ambiguity that your application code must then resolve. Wire the output into a decision harness that can split a single input into multiple downstream workflows, queue a clarification question back to the user, or escalate to a human agent. The next section provides the copy-ready template, and the implementation harness section that follows will show you how to build the validation, retry, and logging layers around it.

This template is designed to be copied directly into your prompt management system or codebase. It forces the model to identify all plausible intents, extract supporting evidence, and make a structured recommendation rather than silently picking the highest-probability match. The square-bracket placeholders represent the dynamic parts you'll inject at runtime: your intent taxonomy, input text, output schema, and operational constraints.

codeCopy
You are an intent classification auditor. Your job is to detect when a user input could reasonably match multiple intent categories and flag the ambiguity before routing occurs.

## INPUT
[USER_INPUT]

## INTENT TAXONOMY
[INTENT_TAXONOMY]

## INSTRUCTIONS
1. Identify every intent from the taxonomy that could plausibly match the input, even partially.
2. For each candidate intent, extract the exact text span from the input that supports the match.
3. Assign a confidence score (0.0 to 1.0) to each candidate based on how strongly the evidence supports that intent.
4. If multiple intents have confidence above [CONFIDENCE_THRESHOLD], set `ambiguity_detected` to true.
5. Recommend a primary intent only if one candidate is clearly dominant. Otherwise, set `primary_intent` to null.
6. If ambiguity is detected and no primary intent can be confidently selected, generate a clarification question that would disambiguate the top candidates.
7. If exactly one intent matches with high confidence, set `ambiguity_detected` to false and `clarification_needed` to false.

## OUTPUT SCHEMA
Return valid JSON matching this structure:
{
  "ambiguity_detected": boolean,
  "candidates": [
    {
      "intent": string,
      "confidence": number,
      "evidence_spans": [string],
      "rationale": string
    }
  ],
  "primary_intent": string | null,
  "clarification_needed": boolean,
  "clarification_question": string | null,
  "routing_decision": "route" | "clarify" | "escalate"
}

## CONSTRAINTS
- Never select an intent not present in the provided taxonomy.
- If no intent matches, return an empty candidates array and set routing_decision to "escalate".
- Evidence spans must be verbatim substrings from the input.
- Do not invent clarification questions that assume facts not present in the input.

To adapt this template, replace [INTENT_TAXONOMY] with your actual taxonomy as a structured list or JSON array. Set [CONFIDENCE_THRESHOLD] based on your tolerance for false disambiguation—start at 0.6 and tune after reviewing production logs. Inject [USER_INPUT] at runtime from your application layer. Before deploying, run this prompt against your golden test set and verify that the JSON output parses correctly, that evidence spans are exact substrings, and that the clarification questions are actionable rather than generic.

This prompt is designed to sit between your initial intent classifier and your downstream routing logic. When a primary classifier returns low confidence or multiple high-scoring intents, this prompt acts as a second-pass disambiguation engine. It should not be called on every input—only on inputs that fail a confidence threshold check or trigger a multi-intent flag from the primary classifier. This keeps latency and cost under control while adding a safety net for the most ambiguous cases.

Wire the prompt into a classification middleware that first calls your primary intent model. If the primary model's top intent confidence is below a configurable threshold (start at 0.75 and tune based on your tolerance for misrouting) or if the top two intents are within a margin (e.g., 0.15), invoke this ambiguity flagging prompt. Pass the original user input as [INPUT], the primary classifier's candidate intents and scores as [CONTEXT], and your full intent taxonomy as [TAXONOMY]. The output should be parsed as JSON and validated against a schema that requires candidate_intents (array of objects with intent, evidence_spans, and confidence), primary_recommendation (string), and clarification_needed (boolean). If clarification_needed is true, route to a human agent or a clarification generation module. If false, route to the workflow associated with primary_recommendation.

Validation and retry logic is critical here. After parsing the JSON output, validate that every evidence_span is a substring of the original input—hallucinated spans are a common failure mode. If validation fails, retry once with a stricter prompt that appends: [CONSTRAINT]: Every evidence_span must be an exact substring of the input text. If no exact match exists, use an empty string. If the retry also fails validation, escalate to human review. Log every ambiguity resolution with the original input, primary classifier scores, disambiguation output, and final routing decision. This trace is essential for tuning your confidence thresholds and detecting when your intent taxonomy has overlapping categories that need refinement.

Model selection matters for this workflow. The prompt requires careful reasoning over evidence spans and taxonomy boundaries, which smaller models often handle poorly. Use a capable model like Claude 3.5 Sonnet or GPT-4o for this disambiguation step, even if your primary classifier uses a smaller, faster model. The cost is justified because this prompt runs only on a small fraction of traffic. For high-throughput systems, consider caching disambiguation results for identical or near-identical inputs using a semantic deduplication layer. Do not use this prompt as your only classification step—it is a safety net, not a replacement for a fast, high-recall primary classifier.

Human review integration is the final piece. When clarification_needed is true, surface the original input, the candidate intents with evidence spans, and the model's primary recommendation in a review queue. The human reviewer should confirm or override the recommendation and optionally update the taxonomy if the ambiguity reveals a genuine gap. Track the rate of clarification_needed flags over time—if it exceeds 10-15% of disambiguation calls, your intent taxonomy likely needs restructuring or your primary classifier threshold is set too low.