Ambiguity Severity Grading Prompt

This prompt is for triage system designers who need a nuanced, multi-level assessment of input ambiguity before routing. A binary ambiguous/not-ambiguous flag is insufficient when downstream actions range from automated resolution to immediate human escalation. This prompt grades ambiguity on a defined severity scale with explicit criteria per level, producing a structured output that a routing engine can use to decide whether to auto-resolve, ask a clarifying question, route to a generalist model, or escalate to a human reviewer. Use this when your system must make differentiated decisions based on how severely an input is underspecified, contradictory, or open to multiple interpretations.

The ideal user is an AI platform engineer or product engineer building a classification and routing middleware layer. You should have a defined taxonomy of downstream handlers—such as specialized models, tool chains, human queues, or clarification modules—and you need a consistent, auditable way to decide which handler receives each input. Required context includes the raw user input, any available conversation history or session state, and the set of possible routing destinations with their capabilities. Do not use this prompt when a simple confidence threshold on a single classification label is sufficient, when all ambiguous inputs should follow the same fallback path, or when latency constraints prohibit the additional inference step that severity grading introduces.

Before deploying this prompt, define your severity levels and their corresponding routing actions. A common three-tier scheme is: low severity (minor underspecification resolvable by inference or default assumptions, route to automated handler), medium severity (genuine ambiguity requiring a clarifying question, route to clarification module), and high severity (contradictory signals, safety implications, or multiple equally valid interpretations, escalate to human review). Each tier needs explicit, testable criteria. For example, 'low severity' might mean exactly one entity is missing but inferrable from context, while 'high severity' might mean the input contains mutually exclusive constraints. Without these operational definitions, the severity grades become arbitrary and your routing engine will make inconsistent decisions. Validate your severity definitions against a golden set of labeled examples before production use, and monitor severity distribution shifts as an early signal of input quality changes or model behavior drift.

This prompt template implements a multi-level ambiguity severity grader. It is designed for triage system builders who need more than a binary ambiguous/clear flag. The prompt instructs the model to analyze an input against defined severity criteria, output a structured grade, cite the specific ambiguous spans, and explain its reasoning. This enables differentiated downstream handling: low-severity ambiguity might auto-resolve, medium severity might trigger a clarification question, and high severity might escalate immediately to a human queue.

textCopy
You are an ambiguity severity grader for a production triage system. Your job is to analyze user inputs and assign an ambiguity severity grade based on the criteria below. You must be precise, consistent, and conservative. When in doubt, grade higher rather than lower.

## INPUT
[USER_INPUT]

## CONTEXT (if available)
[CONVERSATION_HISTORY]
[USER_PROFILE]

## AMBIGUITY SEVERITY SCALE
Grade the input on this 4-level scale:

**SEV0 - Clear**: The input is unambiguous. Intent, entities, and requested action are explicit. No conflicting signals. Example: "Reset my password for account jane@example.com."

**SEV1 - Mild Ambiguity**: The input has one minor underspecified element that can be reasonably inferred from context or defaults. Auto-resolution is acceptable. Example: "What's the status?" when conversation context clearly references a specific order.

**SEV2 - Moderate Ambiguity**: The input has multiple interpretations, missing critical parameters, or conflicting signals. Clarification is required before routing. Example: "Cancel it" when multiple cancellable items exist in context.

**SEV3 - Severe Ambiguity**: The input is fundamentally unclear, self-contradictory, or could map to high-risk actions with very different outcomes. Immediate human escalation is required. Example: "Do the thing we discussed but also the opposite if that doesn't work."

## OUTPUT SCHEMA
Return ONLY valid JSON with this exact structure:
{
  "severity_grade": "SEV0" | "SEV1" | "SEV2" | "SEV3",
  "confidence": 0.0-1.0,
  "ambiguous_spans": ["exact text span 1", "exact text span 2"],
  "possible_interpretations": ["interpretation A", "interpretation B"],
  "missing_information": ["what critical info is absent"],
  "reasoning": "Brief explanation of the grade assignment.",
  "recommended_action": "auto_resolve" | "clarify" | "escalate"
}

## CONSTRAINTS
- If severity is SEV0, ambiguous_spans must be an empty array.
- If severity is SEV3, recommended_action must be "escalate".
- possible_interpretations must list at least 2 distinct interpretations for SEV2 and SEV3.
- missing_information must be non-empty for SEV1 and above.
- confidence reflects your certainty in the grade assignment, not the clarity of the input.
- Do not hallucinate context. Only use provided [CONVERSATION_HISTORY] and [USER_PROFILE].
- If no context is provided, treat the input as standalone.

Adaptation guidance: Replace [USER_INPUT] with the raw text to grade. [CONVERSATION_HISTORY] and [USER_PROFILE] are optional context slots—remove them if your triage system operates on isolated inputs. Adjust the severity scale examples to match your domain's specific ambiguity patterns. The recommended_action field should map to your actual downstream routing logic. For high-risk domains like healthcare or finance, add a [RISK_LEVEL] parameter and tighten the escalation criteria. Always validate the JSON output against the schema before routing; a malformed grade should trigger a retry or fallback to SEV3 escalation.

The Ambiguity Severity Grading Prompt is designed to sit between an input ingestion layer and a routing dispatcher. Its job is to produce a structured severity grade—such as none, low, medium, high, or critical—along with explicit criteria evidence, so downstream systems can decide whether to auto-resolve, ask a clarifying question, or escalate to a human. In practice, this means the prompt should be called after initial intent classification but before any tool dispatch or agent handoff. The output must be machine-readable and deterministic enough to drive conditional logic without a human reading every result.

To wire this into an application, wrap the prompt in a function that accepts the raw user input and an optional context payload (e.g., conversation history, account tier, or prior classification results). The function should: (1) assemble the prompt with the [INPUT] and [CONTEXT] placeholders filled; (2) call the model with response_format set to a strict JSON schema that enforces the severity level enum and required evidence fields; (3) validate the response against that schema immediately—if validation fails, retry once with the error message injected into the prompt as a correction hint; (4) log the raw input, severity grade, evidence, model used, latency, and retry count to your observability platform. For high-stakes domains (healthcare, finance, safety), add a rule that any critical or high severity output bypasses automated routing and creates a human review task with the full grading payload attached. Use a model with strong instruction-following and JSON mode support—GPT-4o, Claude 3.5 Sonnet, or equivalent—and set temperature to 0 or near-zero to reduce grading variance.

Common failure modes to guard against: the model may default to medium severity when uncertain, which masks true ambiguity and causes under-escalation. Mitigate this by adding an eval check that compares severity distributions across a golden test set and flags overuse of the middle tier. The model may also hallucinate evidence for its severity grade—validate that cited spans or reasons actually appear in the input. If you're batching ambiguity grading across high-volume streams, implement a circuit breaker that stops processing if the retry rate exceeds 5% or if validation failures spike, and fall back to a conservative high severity default with human review. Finally, treat the severity rubric itself as versioned configuration: when you update the criteria for each level, run a regression suite against known ambiguous inputs to confirm the new rubric doesn't silently shift grading behavior across your traffic.