Rate Limit Header Parsing and Backoff Prompt Template

This prompt is for integration developers and reliability engineers who are building AI agents that call external APIs and must handle HTTP 429 (Too Many Requests) responses gracefully. The core job-to-be-done is converting raw rate-limit response headers into a compliant, executable backoff schedule and a clear user-facing status message. You should use this prompt when your agent's tool-calling layer receives a 429 response and you need structured, machine-readable instructions for how long to wait, when to retry, and what to tell the user, rather than relying on a naive fixed-delay or immediate crash.

The ideal user has access to the raw response headers from the upstream API, including Retry-After, X-RateLimit-Remaining, X-RateLimit-Reset, or equivalent vendor-specific headers. The prompt requires these headers as input, along with the current timestamp and any hard deadline for the overall task. Do not use this prompt for proactive rate-limit avoidance based on quota counters alone; it is designed for reactive parsing after a 429 has been received. It is also not a substitute for client-side rate limiters or circuit breakers that should operate at the infrastructure layer before the agent's decision loop.

This prompt is most valuable when the backoff behavior must be auditable and consistent across multiple agent instances or tool definitions. By producing a structured output with fields like retry_after_seconds, jitter_range_ms, and recommended_retry_time, you can log the decision, compare it against actual retry timing, and use it as evidence in reliability reviews. The prompt includes explicit jitter implementation to prevent thundering-herd problems when multiple agents retry simultaneously. If your use case involves a hard real-time deadline where any delay means failure, wire the deadline into the [CONSTRAINTS] block and ensure your harness checks the feasible boolean in the output before proceeding.

Avoid using this prompt when the upstream API does not return standard rate-limit headers, or when the failure is not a 429 (e.g., a 503 with no Retry-After). In those cases, use a general tool error handling and fallback prompt instead. Also avoid this prompt for budget-exhaustion scenarios where the issue is cost, not rate; use the Agent Spend Cap Enforcement Prompt for that. Before shipping, validate the output against a set of known header combinations, including missing Retry-After, malformed date formats, and headers that suggest a reset time in the past. The eval harness described later in this playbook will catch these failure modes before they reach production.

This template is designed to be dropped directly into an agent's system instructions or a dedicated tool-call handler. It instructs the model to parse standard rate-limit headers from an HTTP 429 response, compute a retry delay that respects the server's instructions, apply jitter to prevent thundering herd problems, and produce a structured output that your application can use to schedule the next attempt. The template uses square-bracket placeholders for the dynamic parts you control: the raw headers, your maximum wait policy, and your required output schema.

textCopy
You are a rate-limit compliance handler. You receive the headers from an HTTP 429 Too Many Requests response. Your job is to parse the rate-limit directives, compute a compliant retry delay with jitter, and return a structured decision.

## INPUT
<rate_limit_headers>
[RAW_RESPONSE_HEADERS]
</rate_limit_headers>

## CONSTRAINTS
- Respect the server's Retry-After header if present (prefer seconds over HTTP-date).
- If Retry-After is absent, fall back to X-RateLimit-Reset or a default exponential backoff starting at 1 second.
- Apply full jitter: delay = random_between(0, computed_delay).
- Never exceed [MAX_WAIT_SECONDS] seconds total delay.
- If the computed delay exceeds [MAX_WAIT_SECONDS], set the action to ESCALATE and explain why.
- If the server returns a 429 without any parseable rate-limit headers, apply a default delay of [DEFAULT_DELAY_SECONDS] seconds with jitter.

## OUTPUT SCHEMA
Return a single JSON object with these fields:
{
  "action": "WAIT" | "ESCALATE",
  "delay_seconds": number,
  "retry_after_source": "Retry-After" | "X-RateLimit-Reset" | "default" | "none",
  "jitter_applied": true,
  "reason": "string explaining the decision",
  "retry_at_iso8601": "ISO 8601 timestamp of the next allowed retry"
}

## EXAMPLES
Input: Retry-After: 5
Output: {"action": "WAIT", "delay_seconds": 3, "retry_after_source": "Retry-After", "jitter_applied": true, "reason": "Retry-After header specified 5 seconds. Applied full jitter, resulting in 3 seconds.", "retry_at_iso8601": "2025-01-01T00:00:03Z"}

Input: X-RateLimit-Reset: 1700000005 (current time 1700000000)
Output: {"action": "WAIT", "delay_seconds": 2, "retry_after_source": "X-RateLimit-Reset", "jitter_applied": true, "reason": "X-RateLimit-Reset indicates reset in 5 seconds. Applied full jitter, resulting in 2 seconds.", "retry_at_iso8601": "2025-01-01T00:00:02Z"}

Input: No headers, MAX_WAIT_SECONDS=60
Output: {"action": "WAIT", "delay_seconds": 0, "retry_after_source": "default", "jitter_applied": true, "reason": "No rate-limit headers found. Applied default delay of 1 second with full jitter, resulting in 0 seconds.", "retry_at_iso8601": "2025-01-01T00:00:00Z"}

Input: Retry-After: 120, MAX_WAIT_SECONDS=60
Output: {"action": "ESCALATE", "delay_seconds": 120, "retry_after_source": "Retry-After", "jitter_applied": false, "reason": "Server requested 120 second delay, which exceeds the maximum allowed wait of 60 seconds. Escalating for human or system intervention.", "retry_at_iso8601": null}

To adapt this template, replace the square-bracket placeholders with your application's specific values. [RAW_RESPONSE_HEADERS] should be populated with the actual header string from the 429 response before the prompt is sent to the model. [MAX_WAIT_SECONDS] is your system's tolerance for blocking a workflow—set this based on your user experience requirements and queue depth. [DEFAULT_DELAY_SECONDS] is the fallback when no headers are present; 1-5 seconds is typical. If your application uses a different output format, modify the OUTPUT SCHEMA section but keep the decision logic intact. For high-throughput systems, consider caching the parsed delay and only invoking this prompt when a 429 is first encountered, rather than on every retry attempt. Always validate the returned JSON against the schema before using delay_seconds to schedule a retry—a malformed response should trigger an immediate escalation to your error-handling path.

This prompt is designed to sit inside a tool-call wrapper or middleware layer, not as a standalone chatbot. When your agent receives an HTTP 429 (Too Many Requests) or a 503 with a Retry-After header, the raw response headers are passed into this prompt along with the original request context. The prompt's job is to parse the rate limit headers, compute a compliant backoff schedule with jitter, and produce a structured decision that your application code can execute directly—no human interpretation required. The output includes the delay in milliseconds, the next allowed retry timestamp, and a user-facing message if the agent needs to communicate the delay.

Integration pattern: Wrap every outbound tool call in a decorator or middleware function that catches rate-limit responses. Extract the response status code and all relevant headers (Retry-After, X-RateLimit-Remaining, X-RateLimit-Reset, RateLimit-Reset, and any custom vendor headers). Pass these into the prompt as structured JSON in the [RATE_LIMIT_HEADERS] placeholder. The prompt returns a JSON object with fields backoff_ms, retry_at_iso, should_retry, user_message, and jitter_applied. Your harness then: (1) validates the JSON schema, (2) clamps backoff_ms to your configured [MAX_BACKOFF_MS] ceiling, (3) schedules the retry or escalates if should_retry is false, and (4) logs the full decision for observability. Do not let the model directly sleep or schedule—the prompt advises, the harness enforces.

Validation and safety checks you must implement in the harness: confirm that backoff_ms is a positive integer, that retry_at_iso is a valid ISO 8601 timestamp in the future, and that jitter has been applied (the delay should not be an exact multiple of the Retry-After value). If the prompt returns a backoff_ms exceeding your system's maximum retry window, clamp it and log the override. For high-throughput systems, implement a circuit breaker that counts consecutive 429s per endpoint and opens the circuit if the prompt's backoff decisions aren't preventing persistent retry storms. Store each decision in structured logs with tool_name, status_code, retry_after_header, computed_backoff_ms, and jitter_applied for later analysis of rate-limit patterns across providers.

Model choice: This is a low-latency, structured extraction task. Use a fast, cost-effective model (GPT-4o-mini, Claude Haiku, or equivalent) rather than a large reasoning model. The prompt does not require deep reasoning—it requires precise header parsing and arithmetic. Testing: Build an eval harness that feeds known header combinations (exact Retry-After seconds, HTTP-date formats, missing headers, headers from Stripe, OpenAI, AWS, and GitHub APIs) and asserts that backoff_ms is within expected ranges, jitter is present, and should_retry is correct. Include edge cases: Retry-After: 0, missing headers entirely, headers suggesting a permanent block (some APIs use 429 for quota exhaustion with no reset window), and headers with values exceeding your [MAX_BACKOFF_MS]. Your harness should handle the prompt returning malformed JSON by falling back to a default exponential backoff with jitter—never let a parse failure cause a tight retry loop.