Retry Policy and Backoff Design Prompt

This prompt is designed for SREs and backend engineers who need to design or review a message retry strategy before it ships. It acts as a structured design review partner, producing a comprehensive analysis of a proposed or existing retry policy. The output covers critical dimensions such as the chosen backoff algorithm, maximum retry limits, jitter configuration, and integration with circuit breaker patterns. Use it during the design phase of a new service, when auditing an existing policy for resilience gaps, or as part of a formal pre-production resilience review.

To use this prompt effectively, you must provide concrete context. The prompt assumes you can supply the current or proposed policy configuration, including specific parameters like base intervals, max attempts, and jitter strategy. You also need to articulate the system's consistency requirements—specifically whether operations are idempotent—and describe the expected failure modes of downstream dependencies, such as timeouts, rate limiting, or transient network errors. Without this input, the review will be generic and may miss critical failure modes like thundering herd problems or poison message amplification.

Do not use this prompt for real-time incident response. It is a design-time tool, not a replacement for an on-call runbook. It is also fundamentally unsafe to use for systems where retries are inherently dangerous, such as non-idempotent payment capture endpoints or commands that trigger irreversible physical actuation. In these high-risk scenarios, a retry-first strategy can cause duplicate financial transactions or dangerous physical outcomes, and the correct design pattern is a human-in-the-loop reconciliation process, not an automated backoff algorithm. After the review, the next step is to codify the approved policy into your application's resilience library and validate it against the failure modes identified in the prompt's output.

The prompt below is designed to be pasted directly into your AI system. It instructs the model to act as a Site Reliability Engineer reviewing a proposed retry and backoff strategy for an event-driven or distributed system. The template uses square-bracket placeholders for all variable inputs, such as the system description, current policy, and operational constraints. Replace each placeholder with your specific configuration before use.

textCopy
You are a senior SRE reviewing a retry policy and backoff strategy for a distributed system.

Review the following design against best practices for resilience, thundering herd prevention, and poison message mitigation.

SYSTEM CONTEXT:
[SYSTEM_DESCRIPTION]

PROPOSED RETRY POLICY:
[RETRY_POLICY_DETAILS]

OPERATIONAL CONSTRAINTS:
[CONSTRAINTS]

OUTPUT_SCHEMA:
{
  "overall_assessment": "PASS | FAIL | NEEDS_CLARIFICATION",
  "critical_issues": [
    {
      "severity": "HIGH | MEDIUM | LOW",
      "category": "THUNDERING_HERD | POISON_MESSAGE | CIRCUIT_BREAKER | IDEMPOTENCY | TIMEOUT | OTHER",
      "description": "string",
      "recommendation": "string"
    }
  ],
  "backoff_algorithm_review": {
    "algorithm_type": "FIXED | LINEAR | EXPONENTIAL | DECORRELATED_JITTER | OTHER",
    "is_appropriate": "YES | NO | PARTIALLY",
    "reasoning": "string",
    "suggested_parameters": {
      "base_delay_ms": "number",
      "max_delay_ms": "number",
      "jitter_strategy": "FULL_JITTER | EQUAL_JITTER | DECORRELATED_JITTER | NONE"
    }
  },
  "max_retry_review": {
    "current_max_retries": "number",
    "is_safe": "YES | NO",
    "risk_of_amplification": "HIGH | MEDIUM | LOW",
    "suggested_max_retries": "number"
  },
  "circuit_breaker_assessment": {
    "is_configured": "YES | NO",
    "recommendation": "string"
  },
  "eval_checks": {
    "thundering_herd_risk": "HIGH | MEDIUM | LOW",
    "poison_message_amplification_risk": "HIGH | MEDIUM | LOW",
    "idempotency_required": "YES | NO",
    "timeout_configuration_safe": "YES | NO"
  },
  "revised_policy_suggestion": "string"
}

CONSTRAINTS:
- If the proposed policy is missing, assume a default of 3 retries with no backoff and flag it as a critical issue.
- Always recommend decorrelated jitter for distributed systems to prevent thundering herds.
- If a circuit breaker is not configured, flag it as a HIGH severity issue.
- If idempotency is not addressed, flag it as a HIGH severity issue for any system with at-least-once delivery.
- Provide a concrete, revised policy suggestion in the `revised_policy_suggestion` field.

To adapt this template, replace [SYSTEM_DESCRIPTION] with a concise summary of your architecture, including the message broker, consumer groups, and delivery guarantees. Replace [RETRY_POLICY_DETAILS] with your current or proposed configuration, such as "3 retries with a 1-second fixed delay." Replace [CONSTRAINTS] with any operational limits, such as maximum acceptable latency or data loss tolerance. The output schema is designed to be machine-readable for integration into automated design review pipelines. For high-risk systems where data loss or corruption is unacceptable, always route the AI's output to a human reviewer before implementing the revised policy.

This prompt is designed to be integrated into a pre-merge architecture review pipeline, not used as a one-off chat interaction. The primary integration point is a CI check that runs when a pull request modifies retry configuration files (e.g., retry-policy.yaml, circuit-breaker.json, or application config blocks containing backoff parameters). The harness should extract the relevant configuration block, pass it as [POLICY_CONFIG] to the prompt, and treat the structured output as a review finding that can block the merge if critical issues are detected.

The implementation harness requires three components: a configuration extractor that parses the PR diff and isolates retry-related changes, a model invocation layer that sends the prompt with the extracted config and receives the structured JSON output, and a validation and gating layer that enforces review rules. The output schema should include a severity field (CRITICAL, WARNING, INFO) and a recommendation field. Configure the gating layer to block the merge on any CRITICAL finding—such as missing jitter, unbounded retry counts, or absent circuit breaker integration. WARNING findings (e.g., suboptimal backoff multiplier choice) should generate a non-blocking comment on the PR. The harness must also log the full prompt, response, and model version to an audit trail for post-incident review.

For model selection, use a model with strong reasoning capabilities and reliable JSON output (e.g., Claude 3.5 Sonnet or GPT-4o with structured outputs enabled). Set temperature to 0 to maximize consistency across runs. Implement a retry wrapper around the model call itself—if the model returns malformed JSON or fails to include required fields like severity, retry once with a repair prompt that includes the raw output and the expected schema. If the second attempt also fails, escalate to a human reviewer via a PR comment and a Slack notification. Do not silently skip the check. For high-risk systems (payment processing, healthcare, critical infrastructure), always require human approval on any finding before the PR can merge, regardless of severity classification.

Before deploying this harness to production, validate it against a golden dataset of known retry configurations: one with correct exponential backoff and jitter, one with missing jitter, one with unbounded retries, and one with a misconfigured circuit breaker. Confirm that the harness correctly blocks, warns, or passes each case. Monitor the harness in production for false positive rates—if the model consistently flags a team's standard pattern as CRITICAL, adjust the prompt's [CONSTRAINTS] or add a [CONTEXT] block with organization-specific policies. The harness should also track latency; if the model call exceeds five seconds, consider caching common configuration patterns or switching to a faster model for non-critical reviews.