Inferensys

Prompt

Disaster Recovery Runbook Generation Prompt

A practical prompt playbook for using the Disaster Recovery Runbook Generation Prompt in production SRE and platform engineering workflows.
Developer doing prompt engineering on laptop, prompt variations visible on screen, casual coding session.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job, reader, and constraints for the Disaster Recovery Runbook Generation Prompt.

This prompt is designed for SREs and platform engineers who need to convert static system architecture documentation, dependency maps, and failure mode analyses into actionable, step-by-step recovery runbooks. The primary job-to-be-done is transforming known failure scenarios and documented system topologies into structured procedures that an on-call engineer can follow under pressure. Use it when you have a well-understood failure mode—such as a database failover, a region loss, or a critical service degradation—and a complete, up-to-date map of the system's components and their dependencies. The prompt forces the model to produce decision points, expected durations, rollback gates, and dependency ordering rather than vague prose, making the output directly usable in an incident response context.

The ideal user has access to a system architecture diagram, a dependency graph, and a specific failure scenario description. Required inputs include the failure mode, the affected components, their upstream and downstream dependencies, and any known recovery constraints like RTO and RPO. The prompt is not a real-time incident commander; it is a preparation tool for building a library of pre-computed runbooks. Do not use this prompt when the system architecture is undocumented, when you need real-time incident response guidance, or when the failure mode is novel and unexplored. In those cases, the model will lack the necessary grounding and may produce plausible but incorrect procedures. For high-risk production systems, always have a human SRE review the generated runbook for correctness and completeness before it is added to the incident response library.

Before using this prompt, ensure your architecture documentation is current and your dependency map is accurate. The quality of the generated runbook is directly proportional to the quality of the input context. If your system topology has changed, re-run the prompt with the updated documentation. After generating a runbook, test it in a sandbox or staging environment to validate the steps, timings, and decision points. A runbook that has not been tested is a liability, not an asset.

PRACTICAL GUARDRAILS

Use Case Fit

Where this prompt works, where it fails, and the operational prerequisites for generating reliable disaster recovery runbooks from system architecture documentation.

01

Good Fit: Structured Architecture Inputs

Use when: You have current architecture diagrams, service dependency maps, and infrastructure-as-code repositories. The prompt excels at translating explicit system topology into ordered recovery steps. Guardrail: Validate that input documentation is less than 90 days old; stale architecture docs produce dangerous runbooks that reference decommissioned services.

02

Bad Fit: Implicit or Undocumented Systems

Avoid when: System knowledge lives primarily in team members' heads or scattered chat logs. The prompt will hallucinate plausible-sounding recovery steps for undocumented dependencies. Guardrail: Require a completeness checklist before generation—every service, data store, and external dependency must appear in at least one input document.

03

Required Inputs: Dependency Graph and Recovery Objectives

What you need: A service dependency graph (nodes and directed edges), Recovery Time Objectives (RTOs) per component, Recovery Point Objectives (RPOs) per data store, and current health check endpoint definitions. Without these, the prompt cannot determine recovery ordering or validate step completeness. Guardrail: Pre-process inputs through a dependency completeness check before prompt assembly.

04

Operational Risk: Out-of-Order Recovery Steps

Risk: The model may list recovery steps in a sequence that violates dependency ordering—starting a service before its database is restored, or failing over to a region before DNS propagation completes. Guardrail: Post-process the generated runbook through a topological sort validator that checks every step's prerequisites are satisfied by prior steps.

05

Operational Risk: Missing Rollback Gates

Risk: Generated runbooks often describe forward recovery without defining rollback conditions. A step that fails silently leaves operators with no clear path to revert. Guardrail: Add a mandatory rollback gate for every recovery step—define the success criterion, the failure indicator, and the revert action before the runbook is approved for use.

06

Operational Risk: Unverified Duration Estimates

Risk: The prompt may generate expected durations that are optimistic or based on generic assumptions rather than your environment's actual recovery performance. Guardrail: Flag all duration estimates as unverified placeholders. Require operators to replace them with measured values from the last recovery test before the runbook enters production rotation.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt template for generating structured disaster recovery runbooks from system architecture documentation, ready to paste into your prompt layer with square-bracket placeholders replaced.

This prompt template takes system architecture documentation and produces a structured disaster recovery runbook. It is designed for SREs and platform engineers who need to convert architecture diagrams, dependency maps, and infrastructure specs into actionable recovery procedures. The template forces step-by-step sequencing, decision points, expected durations, and rollback gates—not just a narrative summary. Replace each square-bracket placeholder with your actual inputs before sending to the model. The output is structured JSON suitable for ingestion into runbook automation systems or human-readable documentation.

text
You are a senior SRE generating a disaster recovery runbook from system architecture documentation. Your output must be a step-by-step recovery procedure that an on-call engineer can follow under pressure.

## INPUTS
- System Architecture: [SYSTEM_ARCHITECTURE_DOC]
- Dependency Graph: [DEPENDENCY_GRAPH]
- Infrastructure Specs: [INFRASTRUCTURE_SPECS]
- Known Failure Modes: [KNOWN_FAILURE_MODES]
- Recovery Time Objective (RTO): [RTO_MINUTES] minutes
- Recovery Point Objective (RPO): [RPO_MINUTES] minutes
- Runbook Audience: [AUDIENCE_ROLE]

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "runbook_metadata": {
    "generated_at": "ISO timestamp",
    "system_name": "string",
    "rto_minutes": number,
    "rpo_minutes": number,
    "total_estimated_recovery_minutes": number,
    "last_reviewed": null
  },
  "pre_recovery_checks": [
    {
      "check_id": "string",
      "description": "string",
      "command_or_query": "string",
      "expected_result": "string",
      "failure_action": "string"
    }
  ],
  "recovery_phases": [
    {
      "phase_number": number,
      "phase_name": "string",
      "objective": "string",
      "estimated_minutes": number,
      "depends_on_phases": [number],
      "steps": [
        {
          "step_number": "string (e.g., '3.2')",
          "action": "string",
          "command_or_operation": "string",
          "expected_output": "string",
          "success_criteria": "string",
          "failure_criteria": "string",
          "on_failure": "string",
          "rollback_action": "string or null",
          "estimated_minutes": number
        }
      ],
      "decision_points": [
        {
          "condition": "string",
          "options": ["string"],
          "recommended_path": "string",
          "escalation_trigger": "string or null"
        }
      ],
      "rollback_gate": {
        "condition": "string",
        "action": "string",
        "estimated_rollback_minutes": number
      }
    }
  ],
  "post_recovery_validation": [
    {
      "validation_id": "string",
      "description": "string",
      "test_command": "string",
      "expected_result": "string",
      "failure_action": "string"
    }
  ],
  "dependencies": [
    {
      "service_name": "string",
      "recovery_phase": number,
      "external_dependency": boolean,
      "contact_or_escalation": "string"
    }
  ],
  "rollback_plan": {
    "trigger_condition": "string",
    "full_rollback_steps": ["string"],
    "estimated_rollback_minutes": number,
    "data_integrity_check": "string"
  }
}

## CONSTRAINTS
- Steps must be ordered by dependency: no step may reference a resource not yet recovered.
- Every step with a state-changing command must include a rollback_action.
- Decision points must have at least two options and a recommended path.
- Estimated durations must sum to a total within [RTO_MINUTES] minutes; flag any phase that exceeds its allocation.
- External dependencies must include an escalation contact or fallback procedure.
- If the architecture documentation is incomplete for any dependency, mark it with "UNVERIFIED" and note the missing information.
- Do not invent recovery procedures for components not described in the inputs.

## EVALUATION CRITERIA
After generating the runbook, self-check against these criteria and include a "self_eval" object in the output:
{
  "self_eval": {
    "missing_dependencies_detected": ["string"],
    "out_of_order_steps_detected": ["string"],
    "steps_without_rollback": ["string"],
    "unverified_components": ["string"],
    "rto_feasibility": "FEASIBLE|AT_RISK|EXCEEDED",
    "overall_confidence": "HIGH|MEDIUM|LOW"
  }
}

## EXAMPLES
[EXAMPLES]

## TOOLS AVAILABLE
[TOOLS]

## RISK LEVEL
[RISK_LEVEL]

To adapt this template, start by replacing [SYSTEM_ARCHITECTURE_DOC] with your actual architecture documentation—this can be a design doc, an infrastructure-as-code repo summary, or a structured description of services and their dependencies. The [DEPENDENCY_GRAPH] placeholder should receive a directed graph or list of service-to-service calls, including external APIs and data stores. For [KNOWN_FAILURE_MODES], supply the output of a prior FMEA or incident postmortem analysis; if none exists, set this to an empty array and the model will flag missing information in the self-eval. The [EXAMPLES] placeholder can contain one or two well-structured runbook excerpts that demonstrate your organization's preferred level of detail and command style. If you have tool definitions for infrastructure commands, supply them in [TOOLS] so the model can reference real operations rather than inventing placeholder commands. Set [RISK_LEVEL] to HIGH, MEDIUM, or LOW to adjust the thoroughness of rollback gates and escalation triggers. After generation, always run the output through a validation harness that checks for out-of-order recovery steps, missing rollback actions, and unverified dependencies before the runbook reaches an on-call engineer.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Disaster Recovery Runbook Generation Prompt. Each variable must be populated before the prompt is assembled and sent. Missing or invalid inputs are the most common cause of incomplete or unsafe runbooks.

PlaceholderPurposeExampleValidation Notes

[SYSTEM_ARCHITECTURE_DOC]

Source document describing the system topology, component dependencies, data stores, and network boundaries.

architecture-overview.md containing service dependency graph, database clusters, and message queue topologies.

Must be non-empty text. Validate that the document includes explicit component names and dependency descriptions. If null, abort prompt assembly.

[RTO_RPO_TARGETS]

Recovery Time Objective and Recovery Point Objective values for each critical service or data store.

{"payment-service": {"rto": "15m", "rpo": "5m"}, "user-profile-db": {"rto": "1h", "rpo": "1h"}}

Must be valid JSON with service-name keys and numeric-plus-unit values. Reject if RTO exceeds RPO for any service. Validate against [SYSTEM_ARCHITECTURE_DOC] to ensure all critical components have targets.

[FAILURE_SCENARIO]

The specific disaster scenario this runbook addresses, including scope and initial observable symptoms.

Full region loss of us-east-1. All services in the region are unreachable. Monitoring shows zero healthy hosts for 45 seconds.

Must be a concrete scenario, not a generic category. Validate that the scenario is compatible with the architecture described in [SYSTEM_ARCHITECTURE_DOC]. Reject scenarios that describe impossible failure modes.

[RECOVERY_PERSONNEL]

Roles and contact channels for the humans who will execute the runbook, including escalation paths.

Must include at least one contact channel per role. Validate that escalation path terminates. Warn if all contacts share the same domain or channel.

[TOOL_ACCESS_MATRIX]

Tools, dashboards, and CLI access available to recovery personnel during the incident.

{"aws-cli": "read-write", "pagerduty": "admin", "datadog": "read-only", "feature-flags": "admin"}

Must be valid JSON mapping tool names to access levels. Validate that every recovery action described in the generated runbook has a corresponding tool entry. Flag missing tool coverage.

[DEPENDENCY_MAP]

External and internal service dependencies with their criticality and known failure modes.

{"auth-service": {"criticality": "high", "circuit-breaker": "enabled"}, "payment-gateway": {"criticality": "critical", "circuit-breaker": "disabled"}}

Must be valid JSON. Cross-validate against [SYSTEM_ARCHITECTURE_DOC] to ensure no dependency is omitted. Flag any critical dependency with circuit-breaker disabled.

[RUNBOOK_FORMAT_TEMPLATE]

The expected structure for the output runbook, defining required sections and their order.

{"sections": ["impact-assessment", "decision-tree", "recovery-steps", "rollback-gates", "post-recovery-validation"]}

Must be valid JSON with a non-empty sections array. Validate that required sections include decision points and rollback gates. Reject templates that omit post-recovery validation.

[KNOWN_CONSTRAINTS]

Operational constraints that limit recovery actions, such as maintenance windows, data residency, or compliance holds.

["no cross-region data transfer for PII", "maintenance window: Sundays 02:00-04:00 UTC only", "SOC2 audit in progress"]

Must be a non-empty array of strings. Validate that no constraint contradicts the recovery actions implied by [RTO_RPO_TARGETS]. Flag constraints that may block standard recovery paths.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the disaster recovery runbook generation prompt into a reliable application workflow.

This prompt is not a one-shot chatbot interaction; it is a component in a high-stakes operational pipeline. The generated runbook must be treated as a draft that requires human approval before it can be used in a live incident. The harness should enforce a strict generate → validate → review → approve lifecycle. Because the prompt consumes sensitive system architecture documentation, the harness must also ensure that the input context is sanitized and that the output is stored in an access-controlled runbook repository, not a general-purpose chat log.

Wire the prompt into an application with a clear state machine. First, assemble the [SYSTEM_ARCHITECTURE] context by pulling relevant documents from your internal knowledge base or service catalog. Use a retrieval step (RAG) if the architecture is too large for a single context window, but prefer a curated, pre-assembled document to avoid retrieval gaps. After the model returns the runbook, run a structural validator that checks for the required sections (e.g., Recovery Steps, Decision Points, Rollback Gates) and parses the JSON output. If the JSON is malformed or missing required fields, use a repair prompt or retry with a stricter schema instruction. Log every generation attempt, the input context hash, and the validator result for auditability.

The most critical harness component is the eval step. Before the runbook reaches a human reviewer, run automated checks for out-of-order recovery steps (e.g., restoring a database before its dependent services), missing dependency references, and unrealistic expected durations. You can implement these as deterministic rule-based checks on the parsed output or as a secondary LLM-as-judge call with a focused rubric. Flag any runbook that fails these evals and block it from entering the review queue. For the human review stage, present the generated runbook alongside a diff against the last approved version and highlight the specific eval warnings. Only after an authorized SRE approves the runbook should it be published to the runbook store. Avoid the temptation to auto-approve; a hallucinated recovery step in a disaster scenario can extend an outage from minutes to hours.

IMPLEMENTATION TABLE

Expected Output Contract

Defines the required fields, types, and validation rules for the generated disaster recovery runbook. Use this contract to build a parser, validator, or evaluation harness that rejects incomplete or malformed outputs before the runbook reaches an operator.

Field or ElementType or FormatRequiredValidation Rule

runbook_id

string (slug)

Must match pattern ^dr-[a-z0-9-]+$. Reject if missing or non-conforming.

generated_at

ISO-8601 timestamp

Must parse as valid UTC datetime. Reject if future-dated by more than 5 minutes.

system_scope

string

Must contain at least one service or component name. Reject if empty or generic placeholder like 'TBD'.

recovery_objective

object

Must include 'rto_minutes' (integer >= 0) and 'rpo_minutes' (integer >= 0). Reject if either field is missing or negative.

precondition_checks

array of objects

Each object must have 'check_name' (string), 'expected_state' (string), and 'command_or_query' (string). Reject if array is empty or any required field is missing.

recovery_steps

array of objects

Minimum 1 step required. Each step must have 'step_number' (integer, sequential), 'action' (string), 'expected_duration_seconds' (integer >= 0), and 'rollback_gate' (boolean). Reject if step numbers are non-sequential or rollback_gate is missing.

decision_points

array of objects

If present, each object must have 'condition' (string), 'true_branch_step' (integer), and 'false_branch_step' (integer). Reject if branch targets reference non-existent step numbers.

post_recovery_validation

array of objects

Minimum 1 validation check required. Each object must have 'validation_name' (string) and 'success_criteria' (string). Reject if array is empty.

PRACTICAL GUARDRAILS

Common Failure Modes

Disaster recovery runbooks fail when the prompt hallucinates dependencies, skips rollback gates, or generates steps that cannot be executed in the real environment. These cards cover the most common failure modes and how to prevent them before the runbook reaches an on-call engineer.

01

Hallucinated Dependencies

What to watch: The model invents service names, database instances, or API endpoints that do not exist in the actual infrastructure. This turns a recovery runbook into a work of fiction. Guardrail: Require the prompt to reference only dependencies extracted from a provided architecture document or CMDB export. Validate all named resources against a known inventory before accepting the output.

02

Out-of-Order Recovery Steps

What to watch: The runbook restores a dependent service after the service that calls it, or starts traffic routing before the data layer is consistent. This causes cascading failures during an actual recovery. Guardrail: Include a dependency graph in the prompt context and instruct the model to topologically sort recovery steps. Add an eval that checks for prerequisite violations in the generated sequence.

03

Missing Rollback Gates

What to watch: The runbook describes forward recovery steps but omits decision points where the operator should abort and roll back. Without gates, a failing recovery continues blindly. Guardrail: Explicitly require a rollback condition and rollback procedure after every major state-changing step. Test the output by asking: 'If step N fails, does the runbook say what to do next?'

04

Unrealistic Duration Estimates

What to watch: The model generates optimistic time estimates ('database restore: 2 minutes') that do not match production data volumes or network constraints. This leads to violated RTOs and escalations. Guardrail: Provide historical recovery timing data or volume metrics in the prompt. Instruct the model to mark any duration without supporting data as '[ESTIMATE NEEDS VALIDATION]'.

05

Environment Drift Between Docs and Reality

What to watch: The architecture documentation fed into the prompt is stale. The runbook references decommissioned services, old DNS names, or deprecated tooling. Guardrail: Add a freshness check to the prompt harness. Require a last-verified timestamp on all source documents and flag any runbook generated from docs older than a configured threshold for human review.

06

Credential and Access Assumptions

What to watch: The runbook assumes the operator has root access, specific IAM roles, or VPN connectivity that may not be available during a real disaster when the access control plane itself is degraded. Guardrail: Include an explicit 'Required Access and Prerequisites' section in the output schema. Instruct the model to list every credential, role, and network path assumed, and flag any that might be unavailable during the declared failure scenario.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to evaluate the quality of a generated disaster recovery runbook before shipping it to production. Each criterion includes a concrete pass standard, a failure signal to watch for, and a test method that can be automated or performed manually.

CriterionPass StandardFailure SignalTest Method

Step Ordering Correctness

All recovery steps are ordered by dependency; no step references a resource restored in a later step

A step references a database, service, or credential that has not been restored yet

Topological sort check: extract all resource dependencies from each step and verify they appear in prior steps

Decision Point Completeness

Every conditional branch (if/then) includes both success and failure paths with explicit next actions

A decision point has only one outcome path or ends in an implicit dead-end without escalation

Parse all conditional statements; assert each has at least two child steps or an explicit escalation target

Expected Duration Realism

Each step includes an expected duration in minutes; sum of critical path durations falls within documented RTO

Missing duration on any step or total critical path exceeds Recovery Time Objective by more than 20%

Extract all duration fields; compute critical path; compare against [RTO_MINUTES] from input context

Rollback Gate Presence

Every irreversible action (data mutation, DNS cutover, traffic shift) is preceded by an explicit rollback gate with preconditions

An irreversible action appears without a preceding checkpoint, snapshot reference, or rollback instruction

Scan for action verbs in [IRREVERSIBLE_ACTIONS] list; assert each is immediately preceded by a rollback gate step

Dependency Coverage

All systems, services, and data stores listed in [DEPENDENCY_MAP] appear in the recovery sequence

A dependency from the input architecture documentation is absent from the generated runbook steps

Extract all entities from [DEPENDENCY_MAP]; diff against entities referenced in runbook steps; flag missing

Escalation Path Clarity

Every failure branch terminates in either a retry loop with max attempts, a human escalation with role, or a declared abort condition

A failure branch ends with vague language like 'investigate' or 'try again' without bounds or ownership

Regex scan for failure branch endings; assert each matches retry pattern, escalation role pattern, or abort keyword

Runbook Metadata Completeness

Output includes all required fields: runbook_id, version, last_tested_date, owner_team, rto_minutes, rpo_minutes

Any required metadata field is null, empty, or missing from the output

Schema validation against [OUTPUT_SCHEMA]; assert all required fields present and non-null

Cross-Reference Accuracy

All references to external runbooks, playbooks, or documentation use valid identifiers from [REFERENCE_REGISTRY]

A reference ID does not match any entry in the provided reference registry or is hallucinated

Extract all reference IDs; assert each exists in [REFERENCE_REGISTRY] with exact match on identifier

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a single system architecture document as [INPUT]. Remove strict output schema requirements and ask for a narrative recovery plan first. Focus on step completeness and logical ordering rather than exact timing or rollback gates.

Prompt snippet

code
You are an SRE generating a disaster recovery runbook. Given the following system architecture document, produce a step-by-step recovery plan covering major failure scenarios. Prioritize step ordering and dependency logic over precise timing.

[INPUT]

Watch for

  • Missing dependency ordering between recovery steps
  • Skipping rollback or verification gates entirely
  • Overly optimistic recovery time estimates without evidence
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.