Failure Mode Classification Prompt for Escalation Routing

This prompt is designed for reliability engineers and AI operations teams who need to classify AI system failures into a defined taxonomy before routing them to the correct human review queue. The job-to-be-done is not just to detect that something went wrong, but to produce a structured, diagnostic classification that determines who should handle the issue, how urgently, and what context they need. Use this when your production AI pipeline generates errors, unexpected outputs, or anomalous behaviors that require human triage, and you need a consistent, machine-readable classification to drive automated escalation logic.

The ideal user is an engineer integrating this prompt into an incident management or observability platform. The required context includes the raw model input, the model's output or error trace, any tool-call logs, and the expected behavior. The prompt works best when you have a pre-defined failure taxonomy (e.g., 'hallucination', 'refusal', 'tool_error', 'format_violation', 'policy_boundary') and a set of routing destinations (e.g., 'review_queue_urgent', 'review_queue_standard', 'auto_resolve'). The prompt's value is in its consistency: it forces every failure to be tagged with a root cause category, a severity level, and a recommended response, which prevents ad-hoc triage and ensures that high-severity issues like silent failures or policy violations are not lost in a general support queue.

Do not use this prompt as a substitute for a real-time monitoring dashboard or as the sole arbiter of whether to page an on-call engineer. It is a classification step within a larger escalation pipeline, not a replacement for metric-based alerting. Avoid using it for failures that are already well-understood and can be handled by deterministic retry logic or output repair prompts. The prompt is also not designed for user-facing explanations; its output is structured for machine consumption and internal routing. Before deploying, ensure you have a human review process mapped to each classification label and a feedback loop to validate the prompt's accuracy against postmortem root cause labels.

This prompt template is designed to be dropped into an escalation workflow immediately after a failure event is captured. It expects structured input describing the failure, a defined taxonomy of failure modes, and a set of available review queues. The model's job is to classify the failure, assign a severity, and produce a routing decision with enough context for a human reviewer to act without re-investigating the root cause from scratch.

textCopy
You are a failure mode classifier for an AI production system. Your task is to analyze a failure event, classify it into a predefined taxonomy, and route it to the correct human review queue.

## INPUT
Failure Event: [FAILURE_EVENT]
System Context: [SYSTEM_CONTEXT]

## FAILURE MODE TAXONOMY
[FAILURE_TAXONOMY]

## AVAILABLE REVIEW QUEUES
[REVIEW_QUEUES]

## OUTPUT SCHEMA
Return a single JSON object with the following fields:
{
  "classification": {
    "primary_mode": "<exact taxonomy label>",
    "secondary_modes": ["<additional contributing modes>"],
    "confidence": <0.0 to 1.0>,
    "rationale": "<brief explanation of why this classification was chosen>"
  },
  "severity": {
    "level": "<CRITICAL | HIGH | MEDIUM | LOW>",
    "impact_description": "<what was affected and how severely>",
    "user_visible": <true | false>
  },
  "routing": {
    "primary_queue": "<exact queue name from available queues>",
    "secondary_queue": "<fallback queue or null>",
    "urgency": "<immediate | same_day | next_business_day | informational>",
    "required_context": ["<list of evidence or artifacts the reviewer will need>"]
  },
  "diagnostic_summary": "<one-paragraph summary of what happened, suitable for a reviewer reading this cold>"
}

## CONSTRAINTS
- Use only taxonomy labels exactly as written. Do not invent new categories.
- If the failure does not clearly match any taxonomy entry, set primary_mode to "UNCLASSIFIED" and confidence below 0.5.
- For CRITICAL severity, urgency must be "immediate".
- If user_visible is true, include the user-facing symptom in the diagnostic_summary.
- Do not recommend actions outside the available review queues.
- If multiple modes apply, list the dominant one as primary and up to two contributing modes as secondary.

## EXAMPLES
[EXAMPLES]

To adapt this template, replace each square-bracket placeholder with your system's specific taxonomy, queue definitions, and example failure cases. The taxonomy should be a flat or hierarchical list of failure modes your team has observed or anticipates—such as HALLUCINATION_FACTUAL, TOOL_CALL_MALFORMED, REFUSAL_OVERBROAD, or LATENCY_TIMEOUT. The review queues should map to actual teams or individuals with clear ownership. Before deploying, run this prompt against a golden set of 20–50 known failures with postmortem root cause labels and measure classification accuracy. If accuracy on primary_mode falls below 90%, add more few-shot examples covering the misclassified patterns rather than lengthening the instructions.

This prompt is designed to sit at the critical junction between an AI system's failure event and the human review queue. It should be invoked immediately after a failure is detected—whether from a tool error, a guardrail violation, a low-confidence output, or an explicit agent crash. The prompt's job is to classify the failure into a predefined taxonomy and produce a structured routing decision, not to diagnose the root cause or fix the problem. The implementation harness must treat this as a synchronous classification step: the upstream system calls the model, receives the classification payload, and then uses the severity and recommended_queue fields to place the incident into the correct review bucket. Do not use this prompt for real-time intervention; it is a post-failure triage tool.

To wire this into an application, wrap the prompt in a function that accepts a failure_context object containing the raw error, the agent's last state, the user input, and any tool outputs. The function should construct the prompt by injecting these fields into the [FAILURE_CONTEXT] placeholder, along with the [FAILURE_TAXONOMY] that defines your organization's specific failure categories. After calling the model, validate the output against a strict JSON schema that requires failure_class, severity (e.g., CRITICAL, HIGH, MEDIUM, LOW), recommended_queue, and diagnostic_summary. If validation fails, retry once with the validation error appended to the prompt as a [CONSTRAINT] reminder. Log every classification attempt—including validation failures—to your observability platform so you can track classification accuracy against postmortem root cause labels over time. For high-severity classifications (CRITICAL or HIGH), bypass the normal queue and trigger an immediate alert to the on-call channel with the diagnostic_summary included.

Model choice matters here. Use a model with strong instruction-following and JSON mode support, such as gpt-4o or claude-3-5-sonnet, and set temperature=0 to maximize classification consistency. If your failure taxonomy is large or domain-specific, provide 3–5 few-shot examples in the [EXAMPLES] placeholder showing correct classifications for common failure patterns in your system. Do not use this prompt with small or general-purpose models that may hallucinate taxonomy categories. The next step after implementation is to build an eval harness that compares the prompt's classifications against a golden dataset of failures with known root causes, measuring precision and recall per failure class. This eval should run on every prompt change before deployment.