Cost-vs-Accuracy Tool Decision Prompt

This prompt is for AI platform teams who manage a portfolio of overlapping tools and models at different price points. The job-to-be-done is a structured, auditable tool selection that weighs estimated accuracy against per-call cost, producing an explicit trade-off justification rather than a silent default. The ideal user is an engineering lead or platform architect integrating this prompt into a model gateway, a tool dispatch layer, or an agent loop where every tool call has a measurable cost and a measurable quality expectation. Use this prompt when you have at least two candidate tools that can satisfy the same user intent, you can estimate their relative accuracy for the task, and you need the selection rationale to be reviewable by a human or an eval system.

Do not use this prompt when there is only one available tool, when cost is irrelevant, or when accuracy differences between tools are unknown or unmeasurable. It is also the wrong choice for latency-bound routing where p95 response time is the primary constraint—use the Latency-Aware Tool Router instead. This prompt assumes you have already defined tool schemas with cost and capability metadata. If your tool descriptions lack cost annotations, start with the Operational Constraint Declaration Prompt to augment your schemas before wiring this selection prompt into your harness. The prompt is designed for single-step tool selection, not for multi-tool orchestration or cumulative budget tracking across a workflow; for those scenarios, pair it with the Cost Cap Enforcement Prompt.

The prompt requires the following inputs to be useful: a user intent description, a list of candidate tools with their estimated accuracy scores and per-call costs, a declared accuracy threshold or budget cap, and any hard constraints such as required tools or forbidden tools. The output is a structured decision containing the selected tool, the estimated cost, the accuracy justification, and any trade-off notes. Before deploying, build eval checks that detect unnecessary premium-tool selection when a cheaper tool meets the accuracy threshold, and accuracy regression when the selected tool falls below the minimum bar. Wire the prompt into a harness that logs every selection decision with the inputs, outputs, and model version for auditability. If the decision involves write operations or high-cost calls above a configured threshold, route the output to a human approval queue before execution.

This prompt template is designed for AI platform teams managing a portfolio of models and tools at different price points. It instructs the model to evaluate candidate tools not just on semantic fit, but on the explicit trade-off between predicted accuracy and per-call cost. The output includes a structured decision with justification, making the selection rationale auditable and debuggable. Use this template when cost is a first-class constraint and you need the model to justify why a premium tool was chosen over a cheaper alternative.

textCopy
You are a tool selection router. Your task is to choose the best tool for the user's request by balancing estimated accuracy against per-call cost.

## Available Tools
[TOOLS]

## User Request
[INPUT]

## Operational Context
- Budget Cap: [BUDGET_CAP]
- Acceptable Accuracy Threshold: [ACCURACY_THRESHOLD]
- Current Spend: [CURRENT_SPEND]

## Instructions
1. For each candidate tool, estimate the accuracy (0.0-1.0) and note the per-call cost.
2. Eliminate any tool whose estimated accuracy falls below [ACCURACY_THRESHOLD].
3. From the remaining candidates, select the tool that minimizes cost while meeting the accuracy threshold.
4. If the cheapest qualifying tool exceeds the remaining budget ([BUDGET_CAP] - [CURRENT_SPEND]), select the cheapest qualifying tool anyway but flag a budget overrun.
5. If no tool meets the accuracy threshold, select the highest-accuracy tool available and flag a quality risk.

## Output Schema
Return a JSON object with the following structure:
{
  "selected_tool": "string",
  "estimated_accuracy": number,
  "estimated_cost": number,
  "budget_remaining": number,
  "budget_overrun": boolean,
  "quality_risk": boolean,
  "candidates_evaluated": [
    {
      "tool": "string",
      "estimated_accuracy": number,
      "cost": number,
      "disqualification_reason": "string | null"
    }
  ],
  "justification": "string explaining the trade-off decision"
}

## Constraints
- Do not select a tool with estimated accuracy below [ACCURACY_THRESHOLD] unless no tool qualifies.
- Always evaluate all provided tools before making a selection.
- If two tools have identical cost and accuracy, prefer the one with lower latency.
- Never invent tools not present in the [TOOLS] list.

To adapt this template, replace the square-bracket placeholders with your specific context. The [TOOLS] placeholder should contain a structured list of tool definitions, each including a name, description, per-call cost, and any accuracy hints (e.g., 'gpt-4o-search: high accuracy, $0.03/call' vs. 'gpt-4o-mini-search: moderate accuracy, $0.003/call'). The [ACCURACY_THRESHOLD] should be a number between 0.0 and 1.0 that reflects your application's quality floor. For high-risk domains, set this threshold conservatively and always route quality-risk flagged outputs to human review. Before deploying, validate that the model's JSON output matches the schema exactly—malformed JSON in tool selection can break downstream orchestration silently.

This prompt is designed to sit inside a tool selection middleware that evaluates candidate tools before dispatch. The application layer should first assemble the list of candidate tools with their declared cost tiers (e.g., cost_tier: premium, cost_tier: standard, cost_tier: basic) and estimated accuracy ranges. The prompt receives this candidate set along with the user intent and any operational constraints, then returns a structured decision. Do not use this prompt for real-time latency-critical paths where sub-50ms decisions are required; the model inference time alone will exceed that budget. Instead, use it for asynchronous or pre-flight tool selection where the cost-vs-accuracy trade-off justifies a few hundred milliseconds of reasoning.

Validation and retry logic must wrap the model output before any tool is dispatched. Parse the JSON response and validate: (1) the selected_tool exists in the provided candidate list, (2) the estimated_cost falls within the declared cost tier for that tool, (3) the accuracy_estimate is a float between 0.0 and 1.0, and (4) the tradeoff_justification field is non-empty and references at least one constraint from the input. If validation fails, retry once with the validation error injected into the prompt as additional context. If the retry also fails, fall back to a deterministic rule: select the cheapest tool that meets the minimum accuracy threshold, log the fallback event, and surface the degraded decision to the operations dashboard.

Logging and audit are critical because this prompt makes a cost-bearing decision. Every invocation should produce a structured audit record containing: the full candidate tool list with cost tiers, the model's selected tool and justification, the validation result, any retry attempts, and the final dispatched tool. Store these records in your observability pipeline with the decision_id as the primary key. For high-stakes workflows where selecting a premium tool incorrectly could waste significant budget, add a human review gate for decisions where the cost delta between the selected tool and the cheapest viable alternative exceeds a configurable threshold (e.g., $0.50 per call). Route those decisions to a review queue with a short TTL to avoid blocking the user experience.

Model choice matters for this prompt. Use a model with strong structured output and reasoning capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller models that may collapse the trade-off reasoning into a simplistic pattern. If you observe the model consistently favoring premium tools without adequate justification, add a cost-bias penalty to your eval rubric and consider injecting a counterexample into the prompt showing a case where the standard tool was correctly chosen over the premium option. Monitor for accuracy regression by periodically sampling decisions where the basic or standard tool was selected and running a side-by-side comparison against the premium tool's output on the same input, flagging cases where the accuracy delta exceeds your acceptable threshold.