Escalation Policy Failure Mode Analysis Prompt

This prompt is for AI reliability engineers and platform teams who need to systematically analyze where escalation policies break in production. Use it when you have a defined escalation policy—a documented set of rules, thresholds, and routing paths—and need to identify failure modes, root causes, impact severity, and mitigation recommendations before those failures reach users. The prompt produces a structured failure mode inventory that feeds directly into risk assessments, test planning, and policy hardening. It is not a prompt for designing escalation policies from scratch, nor is it for real-time incident response. It assumes you already have a documented escalation policy to analyze and that your goal is proactive reliability engineering, not reactive firefighting.

The ideal user has access to the full escalation policy document, including risk thresholds, routing rules, tool-use authorization boundaries, and human-in-the-loop criteria. The prompt works best when you provide the complete policy text, the operational context in which it runs (e.g., a customer support AI, an autonomous coding agent, a healthcare triage system), and any known production incidents or near-misses. The output is a structured table of failure modes, each with a unique identifier, a description of the failure, the policy clause it violates, root cause categories (such as threshold ambiguity, missing edge case, or instruction conflict), severity ratings, and concrete mitigation steps. This inventory becomes the foundation for your failure mode and effects analysis (FMEA), your regression test suite, and your policy versioning decisions.

Do not use this prompt when you are still drafting the escalation policy itself. For that, start with the Escalation Threshold Definition Prompt Template or the Conditional Autonomy Policy Prompt. Do not use it during an active incident—the output is analytical and structured, not time-sensitive. For real-time escalation decisions, use the High-Risk Transaction Escalation Prompt or the Customer Support Escalation Routing Prompt. Also avoid this prompt if you lack a written policy to analyze; the model will hallucinate failure modes for an imaginary policy, producing plausible-looking but untethered results. Finally, if your goal is to test an existing policy against adversarial inputs rather than analyze its failure modes, use the Escalation Policy Testing Harness Prompt instead. This prompt is for systematic failure mode discovery, not adversarial red-teaming.

This template is designed to be dropped into your AI reliability workflow. It instructs the model to act as an AI Reliability Engineer and systematically decompose a provided escalation policy into a comprehensive failure mode inventory. The prompt enforces coverage across all major failure categories—false negatives, false positives, boundary ambiguity, state corruption, and more—and requires concrete production manifestations and evidence for each finding. Use this as your starting point before adding domain-specific constraints or integrating it into an automated evaluation harness.

markdownCopy
You are an AI Reliability Engineer. Analyze the following escalation policy for failure modes. Produce a structured failure mode inventory with root causes, impact assessments, and mitigation recommendations. Cover all failure categories listed below. For each failure mode, explain how it would manifest in production and what evidence would indicate its occurrence.

**Escalation Policy to Analyze:**
[ESCALATION_POLICY_TEXT]

**Required Failure Categories to Cover:**
[FAILURE_CATEGORIES]

**Output Schema:**
Return a JSON object with a single key "failure_modes" containing an array of objects. Each object must have the following fields:
- "failure_mode_id": string (unique slug, e.g., "FN_MISSED_PII")
- "category": string (must match one of the provided failure categories)
- "description": string (concise description of the failure)
- "root_causes": array of strings (underlying reasons this failure occurs)
- "production_manifestation": string (how an operator would observe this failure in a live system)
- "evidence_indicators": array of strings (logs, metrics, or alerts that would signal this failure)
- "impact_assessment": string (severity and scope of the impact: Critical, High, Medium, Low)
- "mitigation_recommendations": array of strings (concrete steps to prevent, detect, or recover from this failure)

**Constraints:**
- Do not invent failure modes not plausibly supported by the provided policy text.
- If a failure category is not applicable, state "No plausible failure modes identified for this category" and do not fabricate entries.
- For each failure mode, the evidence indicators must be observable in a standard production logging and monitoring stack.
- Prioritize failures that would lead to user harm, regulatory violation, or significant business impact.

To adapt this template, replace the [ESCALATION_POLICY_TEXT] placeholder with the exact policy document, system prompt excerpt, or configuration file you are reviewing. The [FAILURE_CATEGORIES] placeholder should be replaced with a bulleted list of categories relevant to your domain, such as False Negative Escalations, False Positive Escalations, Threshold Boundary Ambiguity, Multi-Turn State Corruption, Tool-Authorization Bypass, and Injection-Induced Escalation Suppression. After running the prompt, validate the output JSON against the schema programmatically. For high-stakes policies in regulated environments, always route the generated failure mode inventory to a human reliability engineer for review and prioritization before accepting it as a complete risk assessment.

This prompt is designed to run as a batch analysis job or an on-demand diagnostic tool, not as a real-time inline check. You feed it a complete escalation policy document and receive a structured failure mode inventory. The primary integration point is a CI/CD pipeline for policy changes, a scheduled reliability review, or a pre-release gate before a new escalation policy ships to production. Do not wire this prompt into a live decision path—it is an offline analysis tool that informs policy improvements, not a runtime safety net.

Model choice and configuration: Use a model with strong reasoning capabilities and a large context window (e.g., GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro). Set temperature to 0.1–0.2 for consistent structured output. Enable JSON mode or structured output if your provider supports it, mapping directly to the [OUTPUT_SCHEMA] defined in the prompt template. Validation layer: After the model returns the failure mode inventory, run a programmatic validator that checks: (1) every failure mode has a non-empty root_cause, impact_assessment, and mitigation_recommendation field; (2) severity ratings are within the defined enum; (3) the total count of failure modes is non-zero and below a reasonable upper bound (e.g., 50) to catch runaway generation. If validation fails, retry once with the same prompt plus the validation error message appended as a [CONSTRAINTS] update. If the second attempt also fails, flag for human review.

Logging and audit trail: Log every invocation with the policy version hash, the full prompt payload, the raw model output, validation results, and any retry attempts. This is critical because failure mode analysis informs safety-critical policy changes. Store logs in an append-only audit store (e.g., a dedicated S3 bucket or an internal compliance database). Human review gate: The output of this prompt should never auto-apply changes to an escalation policy. Route the validated failure mode inventory to a human reviewer (typically an AI safety engineer or reliability lead) who triages findings, dismisses false positives, and accepts actionable items into the policy improvement backlog. The review interface should display each failure mode with its severity, root cause, and mitigation recommendation, plus an accept/dismiss/edit action.

Integration with policy testing: Pair this prompt with the Escalation Policy Testing Harness Prompt from the same content group. After the failure mode analysis identifies gaps, use the testing harness to generate targeted test scenarios that validate whether mitigations actually close those gaps. What to avoid: Do not run this prompt on every minor policy edit—reserve it for major version changes, pre-release reviews, and scheduled quarterly reliability audits. Do not treat the output as exhaustive; the model may miss novel failure modes that require adversarial red-teaming or production incident postmortems to surface. Always combine automated analysis with human judgment and real-world incident data.