Budget-Enforcement Router Prompt with Hard Cost Limits

This prompt is for infrastructure engineers who need a deterministic, auditable routing decision that guarantees a hard cost ceiling is never exceeded. It is designed for multi-model routing middleware where a request must be classified and dispatched to a model tier, and where exceeding a token or dollar budget is a compliance violation, not a performance degradation. Use this prompt when your system must produce a routing decision that includes an early termination trigger, an overage prevention rule, and a complete audit log entry.

The ideal user is an AI platform engineer operating a model gateway or internal router that dispatches requests to multiple model providers or tiers with different per-token pricing. Required context includes a pre-defined model catalog with exact per-token costs, a hard budget cap per request or per session, and a classification of the incoming request's complexity or required capability level. The prompt expects these as structured variables—it does not infer pricing or budget limits from natural language. The output is a machine-readable routing decision, not a recommendation for a human operator. Wire this directly into your gateway's dispatch logic, with the audit log entry written to your observability stack before the selected model is invoked.

Do not use this prompt for soft budget guidance, cost estimation, or latency-optimized routing where budget is a preference rather than a hard constraint. If your system can tolerate occasional budget overruns for quality improvement, use a cost-aware routing prompt with tradeoff weights instead. Do not use this prompt when the model catalog or pricing is dynamic and not available as structured input—the prompt cannot look up live pricing. Do not use this prompt as a standalone cost estimator; it routes, it does not forecast. If you need a preflight cost estimate before invocation, pair this with a cost-estimate preflight prompt and use the estimate as input to the routing decision.

This prompt template is designed to be dropped directly into a model routing middleware layer. It receives a user request, a model catalog with per-token pricing, and a hard cost ceiling. The prompt's only job is to select a model and produce a routing decision that never exceeds the budget. It includes explicit rules for early termination when no model fits within the remaining budget, and it requires a structured audit log entry for every decision. Use this when you need a guaranteed cost ceiling with no exceptions—such as per-request caps in a multi-tenant SaaS product or daily spend limits on an internal AI platform.

textCopy
You are a budget-enforcement router. Your only job is to select a model for the incoming request that maximizes response quality while guaranteeing the total estimated cost stays at or below the hard cost limit. You must never exceed the limit under any circumstance.

## INPUT

User Request:
[USER_REQUEST]

## MODEL CATALOG

Each available model includes its per-token pricing and a relative quality score (1-10).

[MODEL_CATALOG]

Format: model_id | input_cost_per_1k_tokens | output_cost_per_1k_tokens | quality_score | max_context_tokens | avg_output_tokens

## HARD COST LIMIT

Maximum allowed total cost for this request: [HARD_COST_LIMIT]

This is a hard ceiling. If no model can process this request within the limit, you must route to the designated fallback or reject the request. Never exceed this limit.

## ROUTING RULES

1. Estimate the total cost for each candidate model using this formula:
   total_cost = (estimated_input_tokens * input_cost_per_1k / 1000) + (estimated_output_tokens * output_cost_per_1k / 1000)
2. Estimate input tokens from the user request length. Use the model's avg_output_tokens as the output estimate unless the request explicitly demands a longer response.
3. Filter out any model whose estimated total cost exceeds the hard cost limit.
4. From the remaining models, select the one with the highest quality_score.
5. If no model fits within the budget, route to [FALLBACK_MODEL_ID] if it fits. If the fallback also exceeds the budget, reject with code NO_MODEL_WITHIN_BUDGET.
6. If the user request is empty or unprocessable, reject with code INVALID_INPUT.

## OVERAGE PREVENTION

- Round all cost estimates up to 4 decimal places.
- Add a 10% safety buffer to the estimated input tokens before calculating cost.
- If the estimated cost is within 5% of the hard limit, flag the decision with a COST_WARNING.

## OUTPUT SCHEMA

Return a JSON object with exactly these fields:

{
  "decision": "ROUTE" | "REJECT",
  "selected_model": string | null,
  "estimated_cost": number | null,
  "hard_cost_limit": number,
  "budget_remaining": number | null,
  "quality_score": number | null,
  "rejection_code": "NO_MODEL_WITHIN_BUDGET" | "INVALID_INPUT" | null,
  "cost_warning": boolean,
  "audit_log": {
    "timestamp": string,
    "request_id": string,
    "input_token_estimate": number,
    "output_token_estimate": number,
    "candidate_models_evaluated": number,
    "models_rejected_by_budget": array,
    "safety_buffer_applied": boolean,
    "routing_rule_version": "1.0"
  }
}

## CONSTRAINTS

- Do not include explanations outside the JSON.
- Do not invent model pricing or capabilities not in the catalog.
- If the hard cost limit is zero or negative, reject immediately.
- The audit_log must be complete even for rejected requests.

To adapt this template, replace the square-bracket placeholders with your runtime values. [MODEL_CATALOG] should be populated from a configuration store or database, not hardcoded—model pricing changes frequently. [HARD_COST_LIMIT] should come from your budget enforcement system, whether that's a per-request cap, a per-user daily allowance, or a tenant-level spend control. [FALLBACK_MODEL_ID] must reference a model that is always available and cheap enough to serve as a last resort; a lightweight open-weight model running locally is a common choice. The output schema is designed to be machine-readable by your routing middleware, so keep the field names stable across versions. If you change the routing rules, increment the routing_rule_version in the audit log to maintain traceability.

The Budget-Enforcement Router Prompt is designed to sit at a critical decision point in your request pipeline—after input classification but before model invocation. It receives a pre-assembled context containing the user request, available model catalog with per-token pricing, and the hard cost limit. The prompt's job is to return a single routing decision that never exceeds the budget. Implementation requires wrapping this prompt in a harness that enforces the hard limit at the application layer, not just trusting the model's output. The model can recommend a route, but your code must verify the projected cost against the budget before forwarding the request to the selected model.

Wire the prompt into a pre-invocation middleware function. The function should: (1) load the model catalog with current per-token pricing from your configuration store; (2) calculate the input token count for the user request using the target model's tokenizer; (3) assemble the prompt with [HARD_COST_LIMIT], [MODEL_CATALOG], [USER_REQUEST], and [INPUT_TOKEN_COUNT]; (4) call a fast, inexpensive model (e.g., GPT-4o-mini or Claude Haiku) to produce the routing decision; (5) parse the JSON output and validate that estimated_cost <= hard_cost_limit; (6) if validation fails, fall back to the cheapest available model that can handle the request type, log the discrepancy, and increment a budget_override_count metric. Never forward a request to a model whose projected cost exceeds the limit, even if the prompt's output claims otherwise. The application is the final enforcer.

Add retry logic for malformed JSON outputs. If the routing decision fails to parse, retry once with a stricter prompt variant that includes the previous raw output and an explicit instruction to return only valid JSON. If the second attempt also fails, route to the cheapest model and log a routing_parse_failure event. For audit logging, capture the full routing decision payload, the validated cost estimate, the selected model, the hard limit, and whether the decision was accepted, overridden, or fell back. This audit trail is essential for cost attribution, debugging budget breaches, and demonstrating compliance with spend governance policies. Store these logs in a structured format queryable by request ID and time window.

For high-throughput systems, consider caching routing decisions for identical or near-identical requests within a time window. If the same user request pattern appears repeatedly with the same budget constraint, reuse the prior routing decision rather than invoking the router prompt again. Invalidate the cache when the model catalog pricing changes or when the hard limit is adjusted. For multi-step agent workflows, invoke this prompt at each step where a new model call is required, and track cumulative spend against a session-level budget. If cumulative spend reaches 80% of the session budget, inject an additional constraint into the prompt that forces selection of the cheapest viable model for remaining steps. At 95%, terminate the session and return a budget-exhausted response to the user. Do not rely on the model to track cumulative spend—maintain that counter in application state.