Inferensys

Prompt

Cascading Failure Prevention Review Prompt

A practical prompt playbook for using the Cascading Failure Prevention Review Prompt in production AI workflows to identify single points of failure, transitive dependency risks, and amplification loops.
Risk analyst performing AI risk assessment on laptop, risk matrices visible, casual office risk session.
PROMPT PLAYBOOK

When to Use This Prompt

Understand the job-to-be-done, required context, and boundaries for the Cascading Failure Prevention Review Prompt.

This prompt is for SREs and architects who need to analyze a system architecture for cascading failure risks before an incident occurs. It takes a dependency graph or service topology description as input and produces a structured risk assessment identifying single points of failure, transitive dependency chains, retry storm amplification paths, and thundering herd scenarios. Use this during architecture review gates, pre-launch resilience audits, or when onboarding a new service into a critical path.

This is a design-time analysis tool, not an incident response prompt. It assumes you have already mapped your service dependencies and can describe them in text. The prompt works best when you provide a clear, structured description of service-to-service calls, including the direction of dependencies, protocol types (sync/async), and any known retry or timeout configurations. If your dependency graph is incomplete, the analysis will have blind spots—garbage in, garbage out applies here. For live incident triage, use the Operational Incident and Runbook Prompts instead.

Do not use this prompt when you lack a reliable dependency map, when you need real-time failure detection, or when you are looking for a substitute for chaos engineering experiments. The output is a theoretical risk assessment based on the topology you describe; it cannot observe actual runtime behavior. Always pair this analysis with empirical resilience testing before concluding a system is safe. If the prompt flags a critical single point of failure or retry storm risk, treat that finding as a trigger for deeper investigation, not a final verdict.

PRACTICAL GUARDRAILS

Use Case Fit

Where this prompt works and where it does not. Use these cards to decide if the Cascading Failure Prevention Review Prompt fits your current architecture review workflow.

01

Good Fit: Pre-Production Architecture Review

Use when: you have a service dependency graph, defined retry policies, and timeout configurations before a system goes live. Guardrail: Run this review as a gate before load testing and chaos experiments to catch propagation risks early.

02

Bad Fit: Runtime Incident Response

Avoid when: you are in an active incident and need immediate mitigation steps. This prompt produces a structured analysis, not a runbook. Guardrail: Use the Operational Incident and Runbook Prompts pillar for live incidents; apply this review during the postmortem phase.

03

Required Input: Dependency Graph and Policy Configs

Risk: Without a complete service dependency map, retry budgets, and timeout values, the analysis will miss transitive failure paths. Guardrail: Provide a structured dependency graph, circuit breaker thresholds, and retry configurations as input context. Incomplete inputs should trigger an explicit gap report.

04

Operational Risk: False Confidence in Analysis

Risk: The prompt identifies known patterns like retry storms and thundering herds, but cannot discover novel failure modes unique to your business logic. Guardrail: Treat the output as a starting point for human review, not a comprehensive safety proof. Always pair with chaos experiment results.

05

Good Fit: Post-Incident Blast Radius Analysis

Use when: an incident revealed unexpected failure propagation and you need to map the full blast radius. Guardrail: Feed the postmortem timeline and affected services into the prompt to identify amplification loops that were missed in the original design.

06

Bad Fit: Single-Service Resilience Tuning

Avoid when: you are tuning one service's retry policy or circuit breaker in isolation. This prompt analyzes cross-service propagation. Guardrail: Use the Circuit Breaker Design Review or Retry Policy Configuration Analysis prompts for single-service tuning before running this system-wide review.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A ready-to-use prompt for analyzing system architectures to identify and prevent cascading failure risks.

This prompt template is designed to be pasted directly into your AI workflow. It instructs the model to act as a site reliability engineer conducting a cascading failure prevention review. The model will analyze a provided system architecture description to identify single points of failure, transitive dependency risks, and amplification loops like retry storms or thundering herds. To use it, replace the square-bracket placeholders with your specific system context, constraints, and the desired output format.

markdown
You are a senior Site Reliability Engineer (SRE) conducting a cascading failure prevention review.

Your task is to analyze the following system architecture description and produce a structured dependency graph analysis. Your goal is to identify risks that could cause a localized failure to propagate and take down the entire system.

# SYSTEM CONTEXT
[SYSTEM_ARCHITECTURE_DESCRIPTION]

# REVIEW CONSTRAINTS
- Focus on runtime failure propagation, not deployment or CI/CD issues.
- Assume all services are running their expected versions and configurations.
- Prioritize risks that have a blast radius capable of causing a full system outage.
- [ADDITIONAL_CONSTRAINTS]

# ANALYSIS STEPS
1.  **Dependency Graph:** Map out the critical runtime dependencies between the components described in the system context. Identify synchronous vs. asynchronous calls.
2.  **Single Points of Failure (SPOFs):** Identify any component whose failure would immediately halt a critical user-facing workflow.
3.  **Transitive Dependency Risks:** Trace dependency chains to find cases where the failure of a non-critical component can indirectly cause a critical component to fail.
4.  **Amplification Loops:** Identify positive feedback loops, such as retry storms (where retries overload a struggling service) or thundering herds (where many clients simultaneously retry after a cache expires or a circuit breaker opens).
5.  **Mitigation Recommendations:** For each identified risk, propose a concrete, actionable mitigation strategy (e.g., circuit breaker configuration, bulkhead pattern, timeout tuning, idempotency key design).

# OUTPUT FORMAT
You must respond with a JSON object conforming to the following schema:
{
  "dependency_graph": {
    "nodes": [{"id": "string", "type": "service" | "database" | "cache" | "queue" | "external_api"}],
    "edges": [{"source": "node_id", "target": "node_id", "type": "sync" | "async"}]
  },
  "risks": [
    {
      "risk_id": "string",
      "category": "SPOF" | "Transitive Dependency" | "Amplification Loop",
      "severity": "CRITICAL" | "HIGH" | "MEDIUM",
      "description": "A clear, concise explanation of the failure scenario.",
      "affected_components": ["node_id"],
      "trigger_event": "The specific event that initiates the cascading failure.",
      "blast_radius": "The scope of the system impacted if the risk materializes.",
      "recommended_mitigation": "A specific, actionable engineering recommendation."
    }
  ],
  "summary": "A brief executive summary of the top 3 risks and the overall system resilience posture."
}

# RISK LEVEL CONTEXT
[RISK_LEVEL]

# EXAMPLES OF GOOD ANALYSIS
[EXAMPLES]

After pasting the template, the most critical step is to provide a detailed and accurate [SYSTEM_ARCHITECTURE_DESCRIPTION]. The quality of the analysis is directly proportional to the clarity and completeness of this input. Include information on service communication protocols (sync/async), data stores, caching layers, message queues, and external API dependencies. For high-risk production systems, the generated analysis should be treated as a starting point for a formal human-led Failure Mode and Effects Analysis (FMEA), not as the final safety review. Always validate the model's output against your own deep knowledge of the system's runtime behavior and recent incident history.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Cascading Failure Prevention Review Prompt. Each variable must be populated before execution to ensure the model can produce a reliable dependency graph analysis. Missing or malformed inputs will cause the analysis to miss failure modes or produce false positives.

PlaceholderPurposeExampleValidation Notes

[SYSTEM_ARCHITECTURE_DOC]

Complete system design document describing all services, data stores, message queues, and their interactions

Architecture Decision Record v2.3 including service topology diagram and data flow descriptions

Schema check: must contain service names, dependency directions, and communication protocols. Reject if only high-level diagrams without textual descriptions

[DEPENDENCY_GRAPH]

Machine-readable representation of service dependencies with direction, protocol, and criticality metadata

{"service-a": {"depends_on": ["service-b", "cache-c"], "protocol": "gRPC", "criticality": "high"}}

Parse check: valid JSON with required keys 'depends_on', 'protocol', 'criticality'. Reject if missing transitive dependencies or contains circular references without explicit annotation

[RETRY_POLICY_CONFIGS]

Configuration parameters for all retry mechanisms across service boundaries including max attempts, backoff strategy, and timeout windows

Service A to Service B: max_retries=3, backoff=exponential, initial=100ms, max=2s, jitter=true

Schema check: each entry must specify source, target, max_retries, backoff strategy, and timeout. Flag missing jitter configuration as high-risk. Reject if retry budgets exceed downstream capacity

[CIRCUIT_BREAKER_CONFIGS]

Circuit breaker settings for all protected service calls including failure thresholds, timeout windows, and half-open state behavior

Service B circuit breaker: failure_threshold=5, success_threshold=2, timeout=30s, half_open_max_requests=1

Parse check: each entry must include failure_threshold, timeout, and half-open configuration. Flag configurations where timeout exceeds upstream deadline. Reject if circuit breaker is absent for any dependency marked critical

[HEALTH_CHECK_DEFINITIONS]

Liveness and readiness probe configurations for all services including check depth, dependency ordering, and timeout values

Service C readiness probe: HTTP GET /health/ready, checks DB+Cache, timeout=5s, interval=10s, failure_threshold=3

Schema check: each probe must specify check type, dependencies verified, timeout, and failure threshold. Flag probes that don't verify all critical dependencies. Reject if startup probes missing for services with slow initialization

[LOAD_SHEDDING_POLICIES]

Load shedding rules and priority tiers for all API gateways and service entry points

Gateway tier definitions: critical=payment-auth, high=user-profile, normal=search, low=analytics. Shed at 80% capacity

Parse check: must define priority tiers and shedding thresholds. Flag missing critical path identification. Reject if no differentiation between read and write operations in shedding rules

[TIMEOUT_BUDGETS]

End-to-end timeout allocations across service call chains including per-hop deadlines and total request budgets

User request budget: 2000ms. Hop allocations: Gateway=100ms, ServiceA=500ms, ServiceB=800ms, ServiceC=400ms

Schema check: must specify total budget and per-hop allocations. Flag mismatches where downstream timeout exceeds upstream deadline. Reject if any hop has no explicit timeout

[BULKHEAD_BOUNDARIES]

Resource isolation configuration including thread pools, connection pools, and deployment unit partitioning

Service B bulkheads: thread_pool_primary=50, thread_pool_secondary=20, db_connections=30, cache_connections=20

Parse check: each service must define resource pools and limits. Flag shared pools across critical and non-critical paths. Reject if any critical service lacks resource isolation from noisy neighbors

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Cascading Failure Prevention Review Prompt into an SRE or architecture review workflow.

This prompt is designed to be integrated into a pre-production architecture review pipeline, not run as a one-off chat. The primary integration point is a CI/CD or documentation workflow where architecture dependency graphs are updated. The application should programmatically supply the [SYSTEM_TOPOLOGY] and [DEPENDENCY_GRAPH] inputs from a machine-readable source of truth, such as a service catalog, an infrastructure-as-code manifest, or a distributed tracing topology map. Feeding the model a static, human-written description risks staleness and misses transitive dependencies that are only visible in runtime traces.

The implementation harness must validate the model's output against a strict schema before accepting it. Define a JSON Schema for the expected output that includes arrays for single_points_of_failure, transitive_risks, and amplification_loops. Each entry must have a severity enum of CRITICAL, HIGH, or MEDIUM. After the model responds, run a validation pass. If the output fails schema validation, implement a single retry with a repair prompt that includes the raw output and the specific validation errors. If the retry also fails, log the failure and escalate to a human reviewer via a ticketing system; do not silently discard a malformed risk assessment.

For logging and observability, capture the full prompt, the model's raw response, the validation result, and the final structured output as an artifact of the review. This audit trail is critical for post-incident reviews. When wiring this into an application, use a model with strong reasoning capabilities and a large context window, as the dependency graph can be extensive. Do not use a lightweight or fast model optimized for chat; the cost of a missed cascading failure risk is far higher than the inference cost. Finally, never use the model's output to automatically gate a production deployment. The output is an advisory risk assessment that must be reviewed by a human architect or SRE before any deployment decision is made.

IMPLEMENTATION TABLE

Expected Output Contract

Defines the structured fields, types, and validation rules for the cascading failure prevention review output. Use this contract to parse, validate, and integrate the model response into downstream tooling or review dashboards.

Field or ElementType or FormatRequiredValidation Rule

dependency_graph

JSON Object (nodes, edges)

Must parse as valid JSON. Nodes array must contain at least one entry. Edges must reference valid node IDs.

dependency_graph.nodes[].id

string

Non-empty, unique within the nodes array. Must match the regex ^[a-zA-Z0-9_-]+$.

dependency_graph.nodes[].component_name

string

Non-empty. Should match a service or datastore name from the provided [SYSTEM_ARCHITECTURE_CONTEXT].

dependency_graph.edges[].source

string

Must match an existing node.id. Represents the calling or dependent component.

dependency_graph.edges[].target

string

Must match an existing node.id. Represents the component being called or depended upon.

single_points_of_failure

Array of Objects

If empty, the array must be present but empty. Each object must contain a 'component_id' and 'rationale' field.

amplification_loops

Array of Objects

If empty, the array must be present but empty. Each object must contain 'loop_description' and 'trigger_condition' fields.

retry_storm_risk_assessment

Object

Must contain a 'risk_level' field with one of the allowed enum values: ['low', 'medium', 'high', 'critical']. Must contain a 'vulnerable_paths' array.

retry_storm_risk_assessment.risk_level

string (enum)

Must be one of: 'low', 'medium', 'high', 'critical'. If 'critical', the 'vulnerable_paths' array must not be empty.

thundering_herd_risk_assessment

Object

Must contain a 'risk_level' field with one of the allowed enum values: ['low', 'medium', 'high', 'critical']. Must contain an 'affected_resources' array.

mitigation_recommendations

Array of Objects

Must contain at least one recommendation. Each object must have a 'priority' field (integer 1-5) and a 'recommendation' field (non-empty string).

overall_risk_score

integer

Must be an integer between 1 and 10, inclusive. Represents the aggregate system fragility score.

PRACTICAL GUARDRAILS

Common Failure Modes

Cascading failure reviews can produce dangerous false confidence if the prompt misses hidden coupling, shared infrastructure risks, or amplification loops. These cards cover the most common failure modes and how to prevent them before the review output reaches a decision maker.

01

Hidden Transitive Dependencies

What to watch: The prompt identifies direct dependencies but misses transitive chains where Service A depends on B, which silently depends on C. When C fails, the blast radius surprises everyone. Guardrail: Require the prompt to traverse the full dependency graph to depth N+1 and flag any leaf nodes that lack documented resilience patterns.

02

Shared Infrastructure Blind Spot

What to watch: The review treats services as independent but they share a load balancer, message broker, or database cluster. A single infrastructure failure takes down multiple 'independent' services simultaneously. Guardrail: Add a shared-resource enumeration step to the prompt template that maps every component to its underlying infrastructure and flags co-location risks.

03

Retry Storm Amplification

What to watch: The prompt correctly identifies retry policies but fails to model what happens when retries multiply across call chains. Three services with 3 retries each can produce 27x load amplification on the downstream dependency. Guardrail: Include a retry-multiplication calculation in the harness that computes worst-case amplification factors across the call graph and flags any chain exceeding a defined threshold.

04

Thundering Herd After Recovery

What to watch: The review focuses on failure propagation but misses what happens when a recovered service comes back online and every upstream client floods it simultaneously. Guardrail: Add a post-recovery scenario to the prompt that checks for jitter, staggered reconnection, and circuit breaker half-open state behavior before declaring the system resilient.

05

Timeout Mismatch Cascades

What to watch: Service A has a 5-second timeout calling Service B, but B has a 10-second timeout calling C. A's timeout fires before B can respond, leaving B with abandoned work and resource leaks. Guardrail: Require the prompt output to include an end-to-end timeout budget analysis that flags any upstream timeout shorter than a downstream dependency's expected latency.

06

Confidence Without Evidence

What to watch: The model produces a plausible-sounding analysis with high confidence but no grounding in actual architecture documentation, config files, or dependency manifests. Teams act on hallucinated failure paths. Guardrail: Require every identified failure path to cite a specific source artifact, and add a harness check that flags unsupported claims for human verification before the review is accepted.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the Cascading Failure Prevention Review Prompt before shipping it into production. Each criterion targets a specific failure mode identified in the prompt's harness checks.

CriterionPass StandardFailure SignalTest Method

Dependency Graph Completeness

Output identifies all direct and transitive dependencies listed in [SYSTEM_ARCHITECTURE_DOC]. No dependency present in the input is omitted from the graph.

A known dependency from the input document is missing from the generated dependency graph or risk analysis.

Diff the set of components in the input doc against the set of nodes in the output graph. Flag any missing nodes.

Single Point of Failure (SPOF) Detection

Every component with a fan-out of 1 and no redundancy annotation is flagged as a SPOF. No false positives on components with documented redundancy.

A SPOF is not flagged, or a component with an active-active pair is incorrectly marked as a SPOF.

Inject a test architecture with one known SPOF and one known redundant pair. Check for exact match on SPOF list.

Retry Storm Amplification Loop Identification

Output flags any cycle in the call graph where a downstream service retries to an upstream caller, creating a potential amplification loop.

A known retry cycle (A calls B, B retries to A) is not reported in the amplification loops section.

Provide a call graph with a documented retry cycle. Assert the cycle appears in the output's amplification loop list.

Thundering Herd Scenario Flagging

Output identifies any component that fans-in from 3+ upstream callers with synchronized retry or cron-based triggers.

A component with 5 upstream cron-triggered callers is not flagged as a thundering herd risk.

Inject a component with a high fan-in and synchronized trigger description. Verify it appears in the thundering herd section.

Mitigation Recommendation Specificity

Each mitigation recommendation references a concrete pattern (e.g., circuit breaker, jitter, shuffle sharding) and the specific component it applies to.

A mitigation says 'add resilience' without naming a pattern or target component.

Parse the mitigation list. Assert each entry has a non-empty pattern field and a non-empty target component field.

Transitive Dependency Risk Ranking

Output ranks risks by blast radius, where a dependency failure affecting 5+ downstream components is ranked Critical or High.

A dependency with 8 downstream consumers is ranked Low or Medium without justification.

Provide a dependency with a known high blast radius. Assert its risk ranking is Critical or High in the output.

Output Schema Validity

Output is valid JSON matching the [OUTPUT_SCHEMA] with all required fields present and no extra top-level keys.

Output is missing the 'amplification_loops' array or contains an unexpected 'summary' key at the top level.

Validate the raw output against the expected JSON Schema. Reject on any schema violation.

Evidence Grounding

Every risk claim includes a reference to a specific component or connection from the input [SYSTEM_ARCHITECTURE_DOC]. No hallucinated component names.

Output warns about a failure in 'payment-service' but no such component exists in the input document.

Extract all component names from the output. Assert each is a substring or exact match of a component in the input document.

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a single architecture diagram or service map as [SYSTEM_DESCRIPTION]. Skip formal eval harnesses and focus on qualitative review. Replace structured output schema with a free-text request: "List the top 5 cascading failure risks you see."

Watch for

  • Overly broad analysis without specific dependency paths
  • Missing retry storm and thundering herd checks
  • Model may invent dependencies not present in your description
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.