This prompt is designed for Site Reliability Engineers (SREs) and platform engineering teams who need to convert a degradation event in a multi-agent system into a structured, actionable operational runbook. The core job-to-be-done is accelerating the creation of a consistent, reviewable artifact that documents the failure, the response procedure, and the verification steps. Use it when an agent in your chain has failed, timed out, or produced degraded output, and you have a detailed incident record including the failure context, affected components, and available telemetry. It is not a replacement for your incident management process; it is a tool to produce a standardized runbook draft that your on-call team can review, approve, and execute.
Prompt
Graceful Degradation Runbook Prompt

When to Use This Prompt
Defines the job-to-be-done, ideal user, required context, and boundaries for the Graceful Degradation Runbook Prompt.
The ideal user is an engineer who already understands the system architecture and the specific failure but needs to produce documentation that others can follow under pressure. The prompt requires a structured incident record as input, including the failed agent's role, the degradation trigger, the blast radius, and any partial results or error logs. Without this context, the generated runbook will be generic and unsafe. The prompt explicitly instructs the model to mark uncertainty, avoid fabricating recovery steps, and flag sections that require human completion. This makes it suitable for high-severity incidents where procedural errors could compound the original failure.
Do not use this prompt for real-time automated incident response. It is a documentation-generation tool, not a control-plane component. It should never be wired directly to a circuit breaker or retry controller. The output is a human-readable runbook that must be reviewed before execution. Avoid using it for low-severity events where a full runbook is overhead, or for incidents where the root cause is unknown and the prompt would force the model to speculate. In those cases, use a failure mode classification prompt first, then feed the classified incident into this runbook prompt. Always pair the output with a human approval step before the runbook is published to your incident response channel.
Use Case Fit
Where the Graceful Degradation Runbook Prompt delivers reliable value and where it introduces operational risk.
Good Fit: Post-Incident Runbook Generation
Use when: An agent failure, timeout, or tool outage has already occurred and been resolved. The prompt excels at converting structured incident timelines and failure-mode classifications into actionable, step-by-step runbooks. Guardrail: Always feed the prompt a structured incident record, not raw chat logs. The output quality depends on the completeness of the degradation event context.
Bad Fit: Real-Time Incident Response
Avoid when: The degradation is still active and requires immediate human or automated remediation. This prompt is designed for post-hoc analysis and documentation, not for making live circuit-breaker or traffic-routing decisions. Guardrail: Route active incidents to the Circuit Breaker Activation or Agent Timeout Escalation prompts. Use this runbook prompt only during the postmortem phase.
Required Input: Structured Degradation Event
What to watch: The prompt requires a detailed event record including the failure mode classification, affected agents, timeline, partial results captured, and the final resolution path. Missing fields produce generic, untestable runbooks. Guardrail: Validate the input payload against a schema before calling the prompt. Reject or flag records missing the failure classification, timeline, or resolution fields.
Operational Risk: Untestable Procedures
What to watch: The generated runbook may contain steps that cannot be executed in your environment, such as commands for non-existent tooling or references to internal services by the wrong name. Guardrail: All generated runbooks must pass through a human SRE review and a dry-run validation step in a staging environment before being published to the team's runbook repository.
Operational Risk: Stale Runbook Drift
What to watch: The runbook is accurate at the time of generation but becomes dangerous as your agent architecture, tool APIs, or fallback policies change. An outdated rollback step can cause a worse incident. Guardrail: Attach an expiration date and a source event ID to every generated runbook. Integrate the prompt into a recurring audit workflow that flags runbooks older than the system's last architecture change.
Variant: Runbook with Human Handoff
Use when: The degradation event required human intervention. The standard prompt can be extended to generate a runbook that includes structured human-handoff checklists, approval gates, and communication templates for stakeholders. Guardrail: Add a [HANDOFF_CONTEXT] input variable containing the human operator's notes and decisions. This ensures the runbook captures the full socio-technical resolution path, not just the automated steps.
Copy-Ready Prompt Template
A reusable prompt template for generating operational runbooks from degradation events, ready to paste into your orchestration layer.
This prompt template is the core instruction set for generating a structured runbook from a degradation event. It is designed to be wired into an SRE automation pipeline where an event payload (alerts, logs, incident timeline) is provided as input. The prompt enforces a strict output schema, requires step-by-step procedural reasoning, and includes explicit placeholders for the incident context, risk level, and evaluation criteria. Use this template as the starting point for your runbook generation agent; pair it with the implementation harness to add validation, retries, and human approval gates before the runbook is published to your incident response platform.
textYou are an SRE runbook authoring agent. Your task is to generate a clear, actionable operational runbook from a provided degradation event. The runbook must be safe to execute, ordered correctly, and complete enough for an on-call engineer to follow under pressure. ## INPUT [INCIDENT_CONTEXT] ## CONSTRAINTS - Do not invent steps, commands, or metrics not present in the input context. - If information is missing for a required section, mark it explicitly as '[TO BE DETERMINED]' and explain what is needed. - Order all procedural steps by dependency: verification before mitigation, rollback before close. - Flag any step that requires human approval with '[APPROVAL REQUIRED]'. - The risk level for this runbook is [RISK_LEVEL]. For 'high' risk, include a rollback plan for every state-changing step. ## OUTPUT SCHEMA Produce a JSON object with the following structure: { "runbook_id": "string", "generated_at": "ISO 8601 timestamp", "degradation_trigger": { "summary": "string (one-sentence description of the event)", "alert_sources": ["string"], "severity": "critical|high|medium|low", "impact_scope": "string (affected services, users, or regions)" }, "pre_remediation_checks": [ { "step_number": "integer", "instruction": "string", "expected_result": "string", "on_failure": "string (next action if check fails)" } ], "response_procedures": [ { "phase": "detection|containment|mitigation|recovery", "step_number": "integer", "instruction": "string", "owner_role": "string", "approval_required": "boolean", "rollback_step": "string or null" } ], "rollback_plan": { "trigger_condition": "string", "steps": [ { "step_number": "integer", "instruction": "string", "verification": "string" } ] }, "post_degradation_verification": [ { "step_number": "integer", "check": "string", "pass_criteria": "string" } ], "missing_information": ["string (list of fields that could not be populated from input)"] } ## EXAMPLES [EXAMPLES] ## EVALUATION CRITERIA After generating the runbook, self-evaluate against these checks: 1. Are all procedural steps ordered by dependency (verify before act)? 2. Does every state-changing step have a rollback instruction if risk is 'high'? 3. Are there no invented commands, metrics, or service names? 4. Is every 'TO BE DETERMINED' field accompanied by a description of what information is needed? 5. Is the runbook complete enough for an on-call engineer to execute without additional research? If any check fails, revise the runbook before output.
To adapt this template for your environment, replace the square-bracket placeholders with your pipeline variables. [INCIDENT_CONTEXT] should receive a structured payload containing alerts, logs, and timeline data from your observability stack. [RISK_LEVEL] should be set by your incident classifier upstream. [EXAMPLES] should contain one or two few-shot examples of well-formed runbooks from past incidents in your infrastructure, formatted to match the output schema. If you are using a model with a small context window, truncate the examples to the most critical sections—pre-remediation checks and response procedures—and omit the full schema in the examples. Always validate the generated JSON against the schema before forwarding it to your runbook automation platform; a malformed runbook is worse than no runbook at all.
Prompt Variables
Required and optional inputs for the Graceful Degradation Runbook Prompt. Each placeholder must be populated before the prompt is sent to the model. Missing or malformed inputs are the most common cause of incomplete runbooks.
| Placeholder | Purpose | Example | Validation Notes |
|---|---|---|---|
[DEGRADATION_EVENT_SUMMARY] | Concise description of the degradation event including affected service, observed symptoms, and initial impact scope | Payment API latency increased to 4.2s p95; checkout error rate rose to 3.1%; started 14:22 UTC | Must be non-empty and under 500 tokens. Check for timestamp, metric name, and threshold breach. Null not allowed. |
[AFFECTED_COMPONENTS] | List of system components, agents, or services directly and indirectly affected by the degradation | payment-gateway, order-service, fraud-detection-agent, notification-service | Must be a valid list with at least one entry. Each component should match a known service registry ID. Empty list triggers fallback prompt. |
[CURRENT_STATE_SNAPSHOT] | Observability data at the time of degradation: metrics, logs, traces, alert context, and agent statuses | CPU 92%, memory 78%, connection pool exhausted, 412 timeout errors in 5min window | Must include at least one quantitative metric and one log-derived observation. Accepts structured JSON or plain text. Null allowed if no data is available, but runbook will be marked as low-confidence. |
[DEPENDENCY_GRAPH] | Upstream and downstream dependencies for each affected component, including agent-to-agent handoff edges | payment-gateway depends on fraud-detection-agent; order-service depends on payment-gateway; notification-service depends on order-service | Must be parseable as a directed graph. Each edge needs a source and target. Null allowed for single-component degradations, but multi-agent scenarios require this for blast-radius analysis. |
[PREVIOUS_INCIDENT_IDS] | Reference IDs of related past incidents for pattern matching and known-remediation lookup | INC-2024-0421, INC-2024-0387 | Each ID must match incident management system format. Null allowed for first-occurrence events. Duplicate IDs are ignored. |
[SLA_THRESHOLDS] | Service level objectives and error budgets relevant to the degraded components | payment-api: p95 < 500ms, error rate < 0.5%; order-service: availability > 99.9% | Must include at least one threshold with metric name, operator, and value. Null allowed only if no SLAs are defined, but runbook will omit SLA-breach severity classification. |
[ESCALATION_CONTACTS] | On-call rotation, team handles, or escalation policy identifiers for human handoff if automated recovery fails | payments-sre@example.com, slack: #payments-incidents, pagerduty: PAYMENTS-ONCALL | Must include at least one contact channel. Format check: email, Slack channel ID, or PagerDuty schedule ID. Null triggers a warning that runbook will lack human escalation instructions. |
[RUNBOOK_OUTPUT_FORMAT] | Desired structure for the generated runbook: sections, fields, and output schema | JSON with sections: triggers, response-steps, rollback, verification, escalation | Must be a valid schema definition or named preset. Supported presets: 'standard', 'compact', 'incident-command'. Invalid format triggers default schema with a warning. |
Implementation Harness Notes
How to wire the Graceful Degradation Runbook Prompt into an SRE workflow with validation, retries, and human review gates.
This prompt is designed to be called from an incident management platform (e.g., PagerDuty, ServiceNow, or a custom orchestrator) after a degradation event is declared. The application layer must assemble the [DEGRADATION_EVENT_LOG], [AFFECTED_SERVICES], and [CURRENT_INCIDENT_TIMELINE] from monitoring systems, not from free-text operator notes. The prompt expects structured input; feeding it raw, unformatted logs will produce unreliable runbooks. The output should be treated as a draft runbook that requires human approval before it is published to the team's runbook repository or executed in production.
Wire the prompt into a two-stage pipeline. Stage 1: Generation. Call the model with the prompt template, passing the assembled context. Set temperature low (0.1–0.3) to prioritize procedural consistency over creativity. Use a model with strong instruction-following and long-context handling, such as Claude 3.5 Sonnet or GPT-4o. Stage 2: Validation. Parse the generated JSON output and run automated checks before any human sees it: (a) verify the degradation_triggers array is non-empty and each trigger references a specific metric or alert from the input log; (b) confirm response_procedures are ordered and each step includes an owner and expected_duration; (c) check that rollback_instructions exist and reference the specific change or deployment that preceded the degradation; (d) ensure post_degradation_verification contains at least one quantitative check (e.g., 'error rate below 0.1% for 10 minutes'). If validation fails, retry once with the validation errors appended to the [CONSTRAINTS] field. If the retry also fails, escalate to the on-call SRE with the partial output and validation failure summary.
Human review is mandatory before execution. The validated runbook must be surfaced in a review UI that highlights: (a) any step marked with requires_manual_approval: true, (b) rollback instructions that modify production infrastructure, and (c) verification steps that depend on metrics not currently available. Log every generation attempt, validation result, and review decision to the incident timeline for postmortem traceability. Do not automatically execute any runbook step that mutates infrastructure, even if the model marks it as safe. The prompt's [RISK_LEVEL] placeholder should be set by the incident commander, not inferred by the model, and must be high for any degradation affecting customer-facing services. Avoid wiring this prompt into a fully autonomous loop; the cost of a bad runbook executed without review exceeds the latency savings.
Expected Output Contract
Fields, format, and validation rules for the JSON runbook generated by the Graceful Degradation Runbook Prompt. Use this contract to parse, validate, and store the output before presenting it to an on-call engineer.
| Field or Element | Type or Format | Required | Validation Rule |
|---|---|---|---|
runbook_id | string (slug) | Must match pattern ^rd-[a-z0-9-]+$. Generate if absent. | |
degradation_trigger | object | Must contain 'event_type' (string) and 'detection_timestamp' (ISO 8601). 'event_type' must be from the allowed enum: [TIMEOUT, TOOL_FAILURE, AGENT_UNAVAILABLE, CIRCUIT_OPEN, STALE_CONTEXT, PARTIAL_RESULT]. | |
response_procedures | array of objects | Array length >= 1. Each object must have 'step_number' (integer, sequential), 'action' (string, non-empty), 'owner' (string: AGENT or HUMAN), and 'expected_outcome' (string). | |
rollback_instructions | array of objects | If present, each object must have 'condition' (string), 'rollback_action' (string), and 'idempotency_check' (string). If absent, validate that 'response_procedures' contains no state-mutating actions. | |
verification_steps | array of strings | Array length >= 1. Each string must be a non-empty, actionable check. Must include at least one step verifying the original degradation trigger is resolved. | |
blast_radius | object | Must contain 'affected_agents' (array of strings, min 1), 'affected_users' (boolean), and 'data_loss_risk' (string enum: NONE, LOW, MEDIUM, HIGH). | |
escalation_path | object | If present, must contain 'target' (string), 'condition' (string), and 'payload_summary' (string). If absent, validate that no procedure step references an external escalation. | |
post_degradation_notes | string | If present, must be non-empty. If absent, validate that 'verification_steps' includes a step to document the event in the postmortem tracker. |
Common Failure Modes
When generating runbooks from degradation events, these failure modes surface first in production. Each card explains what breaks and how to guard against it before the runbook reaches an on-call engineer.
Procedural Gaps in Step Sequencing
What to watch: The model generates response steps that skip prerequisite checks, reorder dependent actions, or omit verification gates. A runbook that says 'restart the service' before 'check disk space' can cause cascading failures. Guardrail: Add a [STEP_ORDER_CONSTRAINTS] block listing required preconditions per action. Validate output with a dependency graph check that confirms no step references an artifact produced by a later step.
Hallucinated Rollback Instructions
What to watch: The model invents rollback commands, flags, or API endpoints that do not exist in your infrastructure. An operator following a fabricated rollback during an incident compounds the outage. Guardrail: Require every rollback step to cite a source from [KNOWN_RUNBOOK_SOURCES] or [TOOL_SCHEMAS]. Run a tool-call simulation against your actual API spec and flag any command that returns a 404 or unsupported operation error.
Missing Post-Degradation Verification
What to watch: The runbook describes recovery actions but omits the verification steps that confirm the system is healthy before closing the incident. Without explicit health checks, operators may declare victory while downstream services remain degraded. Guardrail: Append a mandatory [VERIFICATION_CHECKLIST] section to the output schema. Validate that every recovery action has a corresponding verification step with a measurable success criterion.
Ambiguous Escalation Triggers
What to watch: The runbook uses vague language like 'if this doesn't work, escalate' without defining what 'doesn't work' means or who to escalate to. During an incident, ambiguity causes decision paralysis. Guardrail: Require each escalation point to specify a concrete condition (e.g., 'if step 3 returns non-200 after 3 retries'), a named escalation target, and the handoff payload format. Test with a condition parser that confirms every trigger is machine-evaluable.
Stale Context from the Degradation Event
What to watch: The model relies on outdated or incomplete event data when the degradation trigger occurred minutes or hours before runbook generation. A runbook built on stale metrics prescribes the wrong remediation. Guardrail: Inject a [CONTEXT_FRESHNESS_CHECK] that timestamps every input source. If any source exceeds a freshness threshold, the runbook must include a 'verify current state' preamble before any action steps. Log staleness warnings in the output metadata.
Overconfident Single-Path Recommendations
What to watch: The model presents one recovery path as certain when the degradation cause is ambiguous. An operator follows the wrong path because the runbook didn't surface uncertainty or alternatives. Guardrail: Require the output to include a [CONFIDENCE_ASSESSMENT] with a differential diagnosis when multiple causes are plausible. For each candidate cause, provide a distinguishing check. If confidence is below threshold, the first step must be diagnostic, not remedial.
Evaluation Rubric
Use this rubric to test the Graceful Degradation Runbook Prompt before deploying it to production. Each criterion validates a critical property of the generated runbook. Run these checks against a golden set of degradation scenarios and recent production incidents.
| Criterion | Pass Standard | Failure Signal | Test Method |
|---|---|---|---|
Trigger Completeness | Runbook lists all degradation triggers from the input event log with exact timestamps and metric thresholds | Missing trigger events; triggers described without measurable thresholds; timestamps absent | Diff input event log against runbook trigger section; check for exact threshold values |
Step Ordering Safety | Response steps are ordered by dependency: containment before diagnosis, diagnosis before remediation, remediation before verification | Verification step appears before remediation; rollback step appears before change step; missing containment step | Parse step list into dependency graph; verify topological sort matches runbook order |
Rollback Completeness | Every change step has a corresponding rollback instruction with exact command or action to reverse the change | Change step without paired rollback; rollback uses vague language like 'undo the change'; rollback references non-existent state | Extract all change steps and rollback steps; verify 1:1 mapping; check rollback commands are executable strings |
Verification Step Presence | Post-degradation verification section includes health check commands, expected outputs, and pass/fail criteria for each affected component | Verification section missing; verification uses only 'check if it works'; no expected output values provided | Count affected components from incident context; verify each has a health check with expected output in verification section |
Escalation Boundary Clarity | Runbook specifies exactly when to stop following the runbook and escalate to a human, with named escalation target and contact method | No escalation trigger defined; escalation says 'if unsure, ask someone'; escalation target is a generic role without contact info | Search runbook for escalation keywords; verify presence of specific condition, named role, and contact channel |
Context Preservation | Runbook includes incident ID, affected service names, start time, and current degradation state in a summary header | Runbook starts with first step without incident context; service names are generic; timestamps missing from header | Check runbook header for required fields: incident_id, affected_services, degradation_start, current_state |
Partial Result Handling | Runbook explicitly marks any incomplete diagnosis or unknown state as 'UNVERIFIED' with confidence level and recommended next action | Unknown state presented as fact; missing sections silently omitted; no confidence indicators on uncertain findings | Search for UNVERIFIED markers; verify every uncertain statement has a confidence label and next action |
Consumer-Facing Language | Runbook includes a user notification template that explains degradation in plain language, expected duration, and workaround if available | User notification uses internal jargon; no duration estimate; no workaround when one exists; notification section missing entirely | Extract user notification section; check for plain-language summary, duration estimate, and workaround presence |
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Adapt This Prompt
How to adapt
Use the base prompt with a single degradation event and lighter validation. Remove the post-degradation verification section and focus on the core trigger-response-rollback flow. Replace [OUTPUT_SCHEMA] with a simple markdown checklist.
Watch for
- Missing rollback steps when the prompt only describes the response
- Overly broad triggers that generate runbooks for non-degradation events
- No safe-ordering check, so steps may appear out of sequence

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us