Cost-Aware Tool Selection System Prompt

This prompt is for platform operators and AI cost engineers who need an agent to choose between multiple available tools by weighing each tool's capability against its estimated invocation cost. The core job-to-be-done is preventing runaway spend in production agent systems where unconstrained tool selection leads to expensive API calls, unnecessary data processing, or premium-tier service usage when cheaper alternatives would suffice. The ideal user is someone embedding cost metadata directly into the agent's decision layer—not just setting hard caps, but making cost a first-class dimension of tool choice alongside accuracy, latency, and capability fit.

Use this prompt when your agent has access to a tool registry that includes cost-per-call estimates, token consumption projections, or tiered pricing metadata. It is most effective in scenarios where multiple tools can satisfy the same intent at different price points—for example, a local embedding model versus a paid vector search API, or a cached database lookup versus a live third-party enrichment call. The prompt requires that cost metadata is available at decision time, either injected into the system prompt as part of the tool descriptions or accessible through a pre-flight cost estimation tool. Do not use this prompt when all tools have identical cost profiles, when cost information is unavailable or unreliable, or when the primary constraint is latency rather than spend.

This prompt is insufficient on its own for hard spend enforcement. It influences tool selection but does not guarantee budget compliance—pair it with a separate spend-cap enforcement mechanism, circuit breakers, and cost logging. It is also not a replacement for rate-limit handling or retry logic. In high-stakes financial or compliance workflows, always route final tool selections through a human approval gate when estimated costs exceed predefined thresholds. Before deploying, validate the prompt against a test harness that simulates budget pressure, tool unavailability, and cost-estimation errors to ensure the agent degrades gracefully rather than defaulting to the most expensive available option.

This template embeds cost metadata directly into the agent's tool-selection reasoning loop. Instead of treating cost as an afterthought, the prompt forces the model to compare candidate tools on both functional fit and estimated spend before emitting a tool call. The placeholders let you inject your own tool catalog, cost model, budget constraints, and risk tolerance without rewriting the selection logic.

textCopy
You are an agent that selects and invokes tools to accomplish user tasks. Before every tool call, you must evaluate candidate tools against two dimensions: functional suitability and estimated cost.

## TOOL CATALOG
[TOOLS]

## COST MODEL
Each tool entry includes an `estimated_cost` field expressed in [COST_UNIT]. This is the expected cost of a single invocation under normal parameters. Actual cost may vary with input size, output size, and server-side pricing.

## BUDGET CONSTRAINTS
- Per-task budget cap: [BUDGET_CAP] [COST_UNIT]
- Per-call soft threshold: [SOFT_THRESHOLD] [COST_UNIT] (exceeding this requires explicit justification)
- Rate limit window: [RATE_WINDOW] with max [RATE_LIMIT] calls per window
- If remaining budget falls below [MIN_REMAINING], prefer the cheapest viable tool or escalate to human review

## SELECTION RULES
1. Identify all tools that can satisfy the current step's functional requirements.
2. Rank them by `estimated_cost` ascending.
3. Select the lowest-cost tool that meets the functional requirement, unless a higher-cost tool provides a materially better outcome (higher accuracy, richer response, lower latency) that justifies the additional spend.
4. If you select a tool above the soft threshold, include a one-sentence justification in your reasoning.
5. If no tool fits within the remaining budget, respond with: `BUDGET_EXHAUSTED: [remaining_budget] [COST_UNIT] remaining, need [required_cost] [COST_UNIT] for cheapest viable tool.` Do not invoke the tool.
6. Track cumulative spend across all calls in the current task. Report it after each call as: `SPEND_TRACKER: [cumulative_spend] / [BUDGET_CAP] [COST_UNIT]`.

## OUTPUT FORMAT
For every tool call, output a JSON block:
{
  "tool": "<tool_name>",
  "arguments": { ... },
  "estimated_cost": <number>,
  "cost_justification": "<optional, required if cost exceeds soft threshold>",
  "cumulative_spend": <number>,
  "budget_remaining": <number>
}

## FALLBACK AND ESCALATION
- If the primary tool fails with a rate-limit error (HTTP 429), apply the backoff specified in the Retry-After header. Do not retry before the window expires.
- If three consecutive calls to the same tool fail, switch to the next cheapest viable alternative.
- If cumulative spend reaches [ESCALATION_THRESHOLD]% of the budget cap, pause and request human approval before proceeding.

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES
[EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with concrete values from your production environment. The [TOOLS] placeholder should contain a structured list of tool definitions, each with a name, description, parameter schema, and estimated_cost field. Derive [COST_UNIT] from your pricing model—use tokens, dollars, credits, or an abstract cost score. Set [BUDGET_CAP] based on observed per-task spend baselines, and tune [SOFT_THRESHOLD] to flag calls that deserve scrutiny without blocking them outright. The [ESCALATION_THRESHOLD] should be high enough to avoid alert fatigue but low enough to prevent budget overruns before a human can intervene. If your tools have widely varying costs, add a cost_tier field (low/medium/high) to the tool catalog and adjust the selection rules to prefer lower tiers when budget pressure is high. Before deploying, run this prompt through a harness that simulates budget exhaustion, rate-limit responses, and tool failures to verify the agent degrades gracefully rather than looping or hallucinating.

The Cost-Aware Tool Selection System Prompt is designed to sit inside an agent's tool-calling step, not as a standalone chat interface. It receives a list of available tools annotated with estimated costs (in tokens, dollars, or latency), the current task context, and any remaining budget constraints. The prompt's job is to return a structured tool selection that balances capability against cost. In production, this prompt is typically invoked by an agent framework (LangGraph, AutoGen, custom loop) immediately after the planner decides a tool call is needed but before the call is executed. The model never directly invokes the tool; it only produces a selection decision that the harness validates and then executes.

Wire this prompt into your agent loop with a pre-execution validation layer. Before executing the selected tool, validate that: (1) the selected tool exists in the available tool registry, (2) the estimated cost does not exceed the remaining session or task budget, (3) the tool's required arguments are present and type-correct, and (4) the selection rationale references actual tool capabilities rather than hallucinated features. If validation fails, log the rejection reason, increment a cost_selection_failure metric, and either retry with the failure message fed back into the prompt or fall back to the cheapest viable tool. For high-stakes domains (finance, healthcare, infrastructure), insert a human approval gate when the selected tool exceeds a cost threshold or when the model's confidence in the selection appears low—detectable by checking if the rationale is vague or contradicts the tool's documented capabilities.

Log every tool selection decision as a structured event: {timestamp, session_id, task_id, available_tools, budget_remaining, selected_tool, estimated_cost, rationale, validation_passed, actual_cost_post_execution}. This log becomes your cost attribution and debugging backbone. Compare estimated_cost against actual_cost_post_execution to detect systematic under- or over-estimation. If the model consistently selects expensive tools when cheaper equivalents succeed, you may need to adjust cost metadata, add few-shot examples of cost-conscious selection, or tighten the budget constraint language. For model choice, prefer models with strong instruction-following and structured output support (GPT-4o, Claude 3.5 Sonnet, or fine-tuned variants). Avoid using this prompt with models that struggle to reason about trade-offs or that hallucinate tool names—test with a golden set of tool registries and budget scenarios before deployment. Implement a circuit breaker: if three consecutive tool selections fail validation, abort the agent step, log the escalation, and surface the decision to a human operator or fallback workflow.