Plan Generation with Rate Limit Awareness Prompt

Use this prompt when you are building an agent or automated workflow that must call external APIs with published rate limits, and naive sequential execution would trigger HTTP 429 errors, throttling, or temporary bans. The core job-to-be-done is generating a multi-step execution plan that respects a known rate limit budget, schedules requests to stay under that budget, and defines explicit backoff behavior when limits are exceeded. The ideal user is an AI engineer or backend developer integrating an LLM-based planner into a production system where API costs, reliability, and predictable execution timing matter. Required context includes the target API's rate limit rules (requests per second, per minute, or per day), the list of planned API calls with their expected endpoints and methods, and any hard deadlines or priority ordering for those calls.

Do not use this prompt when the target API has no documented rate limits, when the execution environment can handle retries transparently through a client library or API gateway, or when the number of API calls is trivially small (fewer than five) and unlikely to trigger throttling. This prompt is also inappropriate for workflows where the primary constraint is monetary cost rather than request frequency; use the Plan Generation with Budget Constraints prompt for token or dollar-budgeted plans. If your system already has a robust queue-and-backoff middleware, this prompt adds unnecessary planning overhead—instead, let the middleware handle rate limits and focus the planner on task decomposition and tool selection.

Before adopting this prompt, confirm that you can programmatically enforce the plan's rate limit schedule in your execution harness. The prompt produces a plan with timing and backoff instructions, but the harness is responsible for respecting those instructions, pausing execution, and handling cases where the model underestimates burst risk. If you cannot enforce delays or queue requests in your runtime, the generated plan will not prevent throttling. Start by testing the prompt against a known rate-limited sandbox API, measure whether the generated schedule stays within the declared budget, and add eval checks for missing retry-after handling and burst patterns before deploying to production.

This template is the core instruction set you will send to the model. It forces the planner to treat rate limits as first-class constraints—not afterthoughts—by requiring explicit budget allocation, backoff strategy selection, and request scheduling before any tool call is approved. The square-bracket placeholders let you inject the specific goal, available tools, rate limit profiles, and output schema without rewriting the planning logic each time.

textCopy
You are a planning agent that generates execution plans for API-dependent workflows. Your plans must respect rate limits, avoid burst patterns that trigger throttling, and include explicit backoff and scheduling strategies.

## INPUTS
- Goal: [GOAL_DESCRIPTION]
- Available Tools: [TOOL_LIST_WITH_ENDPOINTS]
- Rate Limit Profiles: [RATE_LIMIT_SPECS]
- Max Execution Budget: [MAX_BUDGET_IN_SECONDS]
- Output Schema: [OUTPUT_SCHEMA]

## PLANNING RULES
1. For every tool call, calculate the estimated request count and compare it against the provided rate limit profile.
2. If a step would exceed a rate limit window, insert a wait step with the exact duration derived from the `Retry-After` header behavior or the documented reset period.
3. Never schedule more than [MAX_CONCURRENT_REQUESTS] requests in parallel against the same rate-limited endpoint.
4. For endpoints with burst limits, spread requests evenly across the available window rather than front-loading.
5. Include an explicit backoff strategy for 429 responses: exponential backoff with jitter, starting at [INITIAL_BACKOFF_SECONDS] seconds, capped at [MAX_BACKOFF_SECONDS] seconds.
6. If the total estimated execution time exceeds the max budget, insert a checkpoint step that reports progress and requests budget extension before continuing.
7. Flag any step that lacks a documented rate limit as `risk: high` and insert a conservative wait between those calls.

## OUTPUT FORMAT
Return a JSON object matching the provided output schema. Each step must include:
- `step_id`: unique identifier
- `tool_call`: the exact tool and endpoint
- `estimated_requests`: number of API calls this step requires
- `rate_limit_budget_consumed`: how much of the window budget this step uses
- `wait_before_seconds`: delay before executing this step (0 if none)
- `backoff_strategy`: the backoff parameters for this specific endpoint
- `risk_level`: low | medium | high
- `fallback_step_id`: which step to execute if this one fails due to rate limiting

## CONSTRAINTS
- Do not propose plans that exceed the documented rate limits.
- Do not omit backoff strategies for any rate-limited endpoint.
- If the goal cannot be achieved within the rate limits and budget, return a plan that completes as much as possible and explicitly marks remaining work as `deferred`.

To adapt this template, replace each bracketed placeholder with concrete values from your application context. [RATE_LIMIT_SPECS] should include the window size, request cap, and whether the API returns a Retry-After header. [TOOL_LIST_WITH_ENDPOINTS] must map each tool to its specific endpoint so the planner can group calls by rate limit domain. If your execution engine cannot handle dynamic wait insertion, add a constraint that all waits must be expressed as explicit sleep tool calls with validated durations. Before deploying, run this prompt against a set of goals that are known to be impossible within the rate limits—the model should produce a partial plan with deferred markers rather than an impossible full plan or a hallucinated workaround.

The rate-limit-aware plan generation prompt is designed to be called by an agent's planning module before execution begins. It expects a clarified goal, a list of available tools with their documented rate limits, and any known scheduling constraints. The prompt should be treated as a pre-execution planning step—not a runtime decision-maker. Its output is a structured plan that the execution engine will follow, so the harness must validate the plan's schema, rate-limit budgets, and backoff strategies before any API call is made.

Wire the prompt into your application with a validate-then-execute pattern. After receiving the model's plan output, run a schema validator that checks for required fields: steps[], each with tool, arguments, estimated_cost, rate_limit_budget_used, backoff_strategy, and retry_after_handling. Then run domain-specific checks: confirm that no step exceeds the tool's documented rate limit, that burst patterns (e.g., three steps calling the same API within a 1-second window) are flagged, and that every step touching a rate-limited tool includes a retry-after header handler. If validation fails, retry the prompt once with the validation errors injected into [CONSTRAINTS]. If it fails again, escalate to a human operator with the invalid plan and validator output. For model choice, use a capable planning model (e.g., GPT-4o, Claude 3.5 Sonnet) with a low temperature (0.1–0.2) to reduce creative variance in budget allocation. Do not use this prompt with models that lack strong instruction-following for structured outputs.

Log every plan generation call with: the input goal, the tool definitions and their rate limits, the raw model output, the validation result, and whether the plan was accepted, retried, or escalated. This trace is essential for debugging when execution stalls due to rate limiting. Common failure modes include the model ignoring retry-after headers, underestimating burst costs, or producing plans that look valid but exceed rate limits when executed concurrently. Your harness should simulate the plan's rate-limit consumption in a dry-run mode before live execution—multiply each step's rate_limit_budget_used by the expected concurrency factor and compare against the API's documented limits. If the dry run fails, reject the plan and request regeneration with the specific violation details added to [CONSTRAINTS].

For production deployments, integrate this prompt into a planning microservice that exposes a /generate-plan endpoint. The service should accept a goal, tool manifest, and rate-limit policies; return a validated plan or an error; and emit structured logs for your observability stack. Never execute a plan directly from the model output without validation. Rate-limit violations in production trigger throttling, bans, or cascading failures across unrelated agent workflows—the harness is your safety boundary.