Tool Execution Budget and Rate Limit Prompt Template

This prompt is for AI operations engineers and platform developers who need to enforce runtime governance on tool calls made by an LLM assistant. The job-to-be-done is embedding a hard policy into the system prompt that limits the number, frequency, and cumulative cost of tool executions within a single user session, preventing runaway loops, quota exhaustion, and unexpected infrastructure bills. The ideal user is someone integrating an LLM into a production product where tool calls consume billable API credits, database resources, or third-party service quotas, and where silent overconsumption is a direct business risk.

Use this prompt when you have a defined set of tools with known cost weights, a per-session budget ceiling, and a requirement for the assistant to communicate its remaining budget to the user gracefully. It is appropriate for chat-based copilots, coding agents with sandboxed execution, and multi-step research assistants that call external APIs. Do not use this prompt when tool calls are free, unlimited, or managed entirely by external circuit breakers in the application layer. It is also insufficient as the sole control for hard real-time rate limiting; the prompt should be paired with deterministic application-level guards that enforce the budget even if the model ignores or misinterprets the instruction.

Before implementing, define your tool cost table, per-session budget, and the desired degradation behavior—whether the assistant should refuse further calls, offer a degraded path, or escalate to a human. The prompt template expects these as concrete variables. Start by copying the template, filling in your tool definitions and budget parameters, and then run eval checks for budget exhaustion, user communication clarity, and adherence under multi-turn pressure. If the workflow involves high-cost or irreversible tool calls, add a human approval gate before execution, not just a budget check.

This prompt template is designed to be placed in the system instructions of a tool-augmented assistant. It defines a hard governance layer that sits between the model's reasoning and the tool execution layer. The prompt enforces three constraints: a maximum number of tool calls per session, a minimum delay between successive calls, and a cumulative cost ceiling. When a limit is reached, the assistant must stop calling tools and communicate the state clearly to the user. The template uses square-bracket placeholders so you can wire in your own thresholds, tool cost registry, and escalation policy without rewriting the behavioral logic.

markdownCopy
You are an assistant with access to external tools. Your tool usage is governed by the following budget and rate limit policy. You must enforce this policy yourself before every tool call. Do not rely on the user or an external system to stop you.

## Budget and Rate Limits
- **Per-Session Tool Call Budget:** You may make a maximum of [MAX_TOOL_CALLS_PER_SESSION] tool calls in this conversation. This count includes all tool calls, regardless of success or failure.
- **Rate Limit:** You must wait at least [MIN_SECONDS_BETWEEN_CALLS] seconds between the end of one tool call and the start of the next. Do not batch or parallelize calls to circumvent this limit.
- **Cost Threshold:** Each tool has an associated cost in [COST_UNIT]. The costs are defined in the tool cost registry below. You must track the cumulative cost of all tool calls made in this session. You may not exceed a total session cost of [MAX_SESSION_COST] [COST_UNIT].

## Tool Cost Registry
[TOOL_COST_REGISTRY]
<!-- Example:
- search_knowledge_base: 1 credit
- execute_sql_query: 5 credits
- send_email: 10 credits
- generate_report: 15 credits
-->

## Pre-Call Checklist
Before every tool call, you must internally verify:
1. The next call will not exceed the remaining call budget.
2. The required wait time since your last tool call has elapsed.
3. The cost of the intended tool, when added to the current session total, will not exceed the maximum session cost.

If any check fails, **do not call the tool**. Instead, inform the user which limit has been reached and what options are available.

## Budget Exhaustion Behavior
When a limit is reached, you must:
- State clearly which limit was hit (call budget, rate limit, or cost threshold).
- Report the current session usage: calls made, total cost accrued.
- Offer alternative paths that do not require additional tool calls, such as providing a partial answer, suggesting manual steps, or requesting an escalation if [ESCALATION_POLICY] permits.
- Never suggest that the user "try again" to bypass the limit.

## User Communication
- When informing the user of a limit, be concise and helpful. Do not apologize excessively.
- If a user asks to override a limit, state that you are not authorized to do so and refer them to the escalation path defined in [ESCALATION_POLICY].
- Do not reveal the raw numerical thresholds unless the user asks directly about your capabilities.

## Tracking
Maintain an internal counter for tool calls and cumulative cost. You may output a brief summary of remaining budget when asked, but do not include it in every response.

To adapt this template, replace the square-bracket placeholders with your production values. The [TOOL_COST_REGISTRY] should be a structured list mapping each tool name to its cost. If you use a dynamic cost model, replace the static registry with a reference to a tool that the assistant can call to look up costs—but be aware that the lookup itself consumes budget. The [ESCALATION_POLICY] placeholder should contain a short instruction, such as "ask the user to contact their admin" or "automatically open a human review ticket." Test the adapted prompt with sessions that intentionally exhaust each limit to confirm the assistant stops calling tools and communicates correctly. If your application layer also enforces these limits, treat the prompt as a first line of defense that provides a better user experience, not as the sole enforcement mechanism.

The Tool Execution Budget and Rate Limit prompt is not a standalone artifact; it is a policy layer that must be enforced by the application runtime. The prompt instructs the model to track its own tool calls and respect limits, but the application must act as the authoritative source of truth. Implement a server-side counter that increments before each tool call is dispatched and decrements the model's declared remaining budget. If the model attempts to exceed the budget, the application should reject the tool call and inject a system message informing the model that the budget is exhausted, rather than relying solely on the model's self-restraint.

Build the harness around three core components: a budget state object, a pre-flight check middleware, and a budget-exhaustion handler. The budget state object should track max_calls, calls_made, reset_condition (e.g., per-session, per-hour sliding window), and graceful_degradation_mode. The pre-flight middleware intercepts every tool call request from the model, checks calls_made < max_calls, and either forwards the call or triggers the exhaustion handler. The exhaustion handler should inject a structured system message like SYSTEM: Tool call budget exhausted. [calls_made] of [max_calls] used. You may only respond with a summary of what was accomplished and guidance for the user on how to proceed. This message must be prepended to the context window before the next inference call to prevent the model from hallucinating additional tool calls.

Logging is critical for debugging budget enforcement. For every tool call, log a structured record containing timestamp, session_id, tool_name, call_number, budget_remaining, model_declared_remaining, and action_taken (allowed, denied, exhaustion_injected). Discrepancies between budget_remaining and model_declared_remaining indicate the model has lost count and the application should correct it via a system message. For rate limits, implement a token-bucket or sliding-window algorithm at the application layer and expose the current state to the model through a [RATE_LIMIT_STATUS] placeholder in the prompt that gets populated at inference time with values like "3 calls remaining in this 60-second window; next reset at 14:32:15 UTC." Never trust the model to enforce time-based rate limits on its own.

When deploying this prompt in production, pair it with an eval harness that tests budget exhaustion behavior explicitly. Create test scenarios where the budget is 0 from the start, where the budget is exhausted mid-task, and where the budget is sufficient. Assert that the model never makes a tool call when calls_made >= max_calls, that the exhaustion message is clear and actionable, and that the model does not fabricate tool results. For high-stakes workflows, add a human approval gate that triggers when the budget is exhausted but the task is incomplete, offering the user the option to extend the budget with an explicit confirmation step rather than leaving the conversation in a broken state.