Ambiguous Intent Human Review Routing Prompt

This prompt is for trust-and-safety engineers and platform operators who need to classify user requests that fall into a gray area. When a request is not clearly safe and not clearly a policy violation, the system must decide whether to respond, refuse, or route to a human review queue. Misclassifying ambiguous intent creates two costly outcomes: unsafe responses that bypass policy, or unnecessary review queue volume that overwhelms human reviewers. This playbook provides a structured classification prompt that scores ambiguity, assigns a risk category, and recommends a review queue with a calibrated confidence level.

Use this prompt when your existing safety classifiers return low-confidence scores, when a user challenges a refusal, or when a request contains mixed signals that could indicate either a legitimate need or a probing attempt. The prompt expects several inputs to function correctly: the raw user request, the session history leading up to the ambiguous turn, the specific policy text that is potentially in conflict, and the confidence scores from any upstream classifiers. Without these inputs, the prompt cannot distinguish between a confused legitimate user and a sophisticated adversarial probe. Wire this prompt into the escalation path between automated safety checks and final human review, not as a replacement for either layer.

Do not use this prompt for clear-cut policy violations that should be refused outright, or for obviously benign requests that should be handled automatically. If you route every low-confidence decision through this prompt without first checking whether the request is trivially safe or trivially unsafe, you will add latency and cost without improving safety outcomes. The prompt is designed for the narrow band of cases where ambiguity is genuine and the cost of misclassification is high. Before deploying, calibrate the upstream confidence thresholds that feed into this prompt, and measure both the false-positive escalation rate and the false-negative pass-through rate against your organization's risk tolerance.

This template is the core instruction set for an AI system that must decide whether an unclear user request can be handled automatically or requires human review. It is designed for trust-and-safety engineers who need to route ambiguous inputs—where the user's goal is not obviously benign or malicious—to the correct queue without over-escalating safe traffic or under-escalating risky content. Copy the template below, replace every square-bracket placeholder with your application's specific data, and test the output against your routing logic before deployment.

textCopy
You are an intent classification and routing assistant for a production AI platform. Your task is to analyze a user request that has been flagged as ambiguous—meaning its intent is unclear, could be interpreted in multiple ways, or contains mixed signals.

## INPUT
User Request: [USER_INPUT]
Conversation History (last 5 turns): [CONVERSATION_HISTORY]
User Role/Context: [USER_CONTEXT]

## CLASSIFICATION TASK
Analyze the request and produce a structured routing decision. Consider:
1. The most likely benign interpretation.
2. The most likely harmful or policy-violating interpretation.
3. The ambiguity factors present (e.g., vague language, missing context, sarcasm, coded terms).

## RISK CATEGORIES
Use the following risk taxonomy:
- `low_risk`: Likely benign, safe to handle automatically.
- `medium_risk`: Potentially problematic, requires review but not urgent.
- `high_risk`: Likely policy violation, safety concern, or regulated domain overstep.
- `critical_risk`: Imminent harm, child safety, self-harm, or security breach.

## REVIEW QUEUES
Available queues:
[REVIEW_QUEUES]

## OUTPUT SCHEMA
Return a single JSON object with this exact structure:
{
  "classification": {
    "primary_intent": "string (best guess at what the user wants)",
    "ambiguity_type": "string (e.g., 'vague_language', 'coded_terms', 'mixed_signals', 'missing_context')",
    "risk_category": "low_risk | medium_risk | high_risk | critical_risk",
    "confidence_score": 0.0-1.0
  },
  "routing": {
    "recommended_queue": "string (must match an available queue name)",
    "escalation_priority": "normal | elevated | urgent",
    "auto_handle": true/false
  },
  "rationale": {
    "benign_interpretation": "string",
    "harmful_interpretation": "string",
    "key_ambiguity_factors": ["string"],
    "decision_reasoning": "string"
  },
  "reviewer_context": {
    "summary_for_reviewer": "string (concise summary if routed to human)",
    "evidence_snippets": ["string (relevant excerpts from input or history)"],
    "suggested_questions": ["string (what the reviewer should investigate)"]
  }
}

## CONSTRAINTS
- If confidence_score is below [CONFIDENCE_THRESHOLD], auto_handle must be false.
- If risk_category is 'high_risk' or 'critical_risk', auto_handle must be false regardless of confidence.
- Never include PII from the input in the rationale or reviewer_context fields unless it is essential to the review and the review queue is authorized to receive it.
- If the request involves [REGULATED_DOMAINS], escalate to the appropriate regulated-domain review queue.
- Do not attempt to fulfill the user's request. Only classify and route.

## EXAMPLES
[FEW_SHOT_EXAMPLES]

After copying the template, replace the placeholders with your operational data. [REVIEW_QUEUES] should be a list of your actual queue names with brief descriptions so the model can select accurately. [CONFIDENCE_THRESHOLD] is a number between 0.0 and 1.0 that gates automatic handling—start at 0.85 and adjust based on your false-positive tolerance. [REGULATED_DOMAINS] should list the specific domains (e.g., 'medical', 'legal', 'financial') that trigger mandatory escalation in your jurisdiction. [FEW_SHOT_EXAMPLES] is critical: provide 3-5 examples showing ambiguous inputs and correct routing decisions, including at least one edge case where the model should escalate despite low apparent risk. Without good examples, the model will default to overly cautious or overly permissive routing. Validate the output JSON against your schema before ingesting it into your routing pipeline, and log every decision where confidence_score falls below 0.7 for offline review and threshold calibration.

This prompt is designed to sit between your primary safety classifier and your human review routing system. It activates only when the classifier returns a low-confidence or ambiguous result—typically a confidence score between 0.4 and 0.7 on a 0–1 scale. Do not run this prompt on every user input; that defeats the purpose of having a fast upstream classifier and will inflate your review queue with clear-cut cases. Instead, gate execution on a confidence threshold check: if classifier.confidence < [AMBIGUITY_THRESHOLD], invoke this prompt with the original user input, the classifier's top-k labels and scores, and any relevant conversation context.

Wire the prompt into an asynchronous decision pipeline. The caller should be a lightweight service that: (1) receives the ambiguous classification event, (2) assembles the prompt with [USER_INPUT], [CLASSIFIER_LABELS], [CONVERSATION_CONTEXT], and [REVIEW_QUEUES], (3) sends the request to your model endpoint with temperature=0 and a structured output mode if available, (4) validates the response against the expected output schema before routing, and (5) logs the full decision for audit. If the model returns a malformed response—missing the risk_category field, an unrecognized review_queue, or a confidence value outside 0–1—retry once with a stricter schema reminder appended to the prompt. If the retry also fails, default-route to the general_review queue and flag the event for manual triage.

The output schema must include at minimum: risk_category (enum of your defined categories), confidence (0–1 float), recommended_queue (must match a valid queue ID from your routing system), reasoning_summary (one sentence for the reviewer), and escalation_urgency (enum: standard, priority, critical). Validate all fields before the routing decision is executed. For escalation_urgency = critical, bypass normal queue assignment and trigger your incident response notification path. Log every routing decision with the model's raw output, the validated fields, the final queue assignment, and a timestamp. This log becomes your primary artifact for measuring routing accuracy, reviewer workload, and threshold calibration over time.

Model choice matters here. Use a model with strong instruction-following and low refusal tendency on safety-adjacent content—Claude 3.5 Sonnet, GPT-4o, or Gemini 1.5 Pro are reasonable defaults. Avoid models that over-refuse on ambiguous safety content, as they will route too many cases to review. If you are running locally or air-gapped, test open-weight models (Llama 3.1 70B+, Qwen 2.5 72B) against a golden eval set before deploying; smaller models often struggle with the nuanced distinction between ambiguous_high_risk and ambiguous_low_risk categories. Do not use this prompt with a model that lacks structured output support unless you add a post-processing repair step.

Build an eval harness before you ship. Create a golden dataset of 200–500 ambiguous inputs with known-correct routing decisions, covering each risk category and urgency level. Run the prompt against this set weekly and track: routing accuracy (did it pick the right queue?), over-escalation rate (routing to human review when the case was actually low-risk), under-escalation rate (routing to low-priority when it should have been critical), and schema compliance rate. Set alert thresholds on over-escalation above 15% and under-escalation above 2%. Feed reviewer corrections back into the eval set: when a human reviewer overrides the routing decision, capture that override as a new labeled example. This closes the loop between prompt behavior and operational reality.