Model Tier Selection Prompt Based on Complexity and Budget

This prompt is designed for infrastructure engineers operating multi-model routing middleware. Use it when you need to decide per-request whether a task requires a premium model, a standard model, or can be handled by a lightweight model. The prompt assesses input complexity against a defined rubric and selects the cheapest model tier capable of producing an acceptable result. This is not a general-purpose chatbot prompt. It belongs inside a routing layer that intercepts requests before they reach your primary model invocation. The output is a structured tier selection with justification, not a conversational response. Deploy this when your system has at least two model tiers with distinct pricing and capability profiles, and when over-routing to premium models is measurably increasing your inference costs without proportional quality gains.

The ideal deployment scenario involves a middleware service that inspects every incoming request before model dispatch. The prompt expects a well-defined model catalog with per-token pricing, capability descriptions, and complexity thresholds. You must provide a complexity rubric that maps input characteristics—such as reasoning depth, domain specificity, output precision requirements, and ambiguity level—to model tier requirements. Without this rubric, the prompt will produce inconsistent or overly conservative routing decisions. The prompt is stateless and should be evaluated per-request, not per-session. It works best when paired with a lightweight pre-classifier that extracts complexity signals before the routing decision, reducing the token overhead of the routing prompt itself.

Do not use this prompt when you have a single model tier, when all requests require premium quality guarantees, or when your cost structure makes routing overhead more expensive than always using a mid-tier model. Avoid deploying it in systems where latency from an additional routing step exceeds the latency savings from using a faster model. If your complexity rubric is poorly calibrated, the prompt will either over-route to premium (wasting money) or under-route to lightweight models (degrading quality). Start with a shadow deployment that logs routing decisions without enforcing them, then compare actual quality outcomes before cutting over. For high-stakes domains such as healthcare, legal, or safety-critical systems, always require human review of routing decisions that downgrade from premium, and never allow automatic downgrades for requests classified as high-risk.

The following prompt is designed to be pasted directly into your routing middleware. It instructs the model to act as a cost-aware router, classifying an incoming request's complexity and selecting the most appropriate model tier from a predefined catalog. The prompt uses a strict rubric to justify its selection, ensuring that a premium model is only used when the complexity genuinely warrants the additional expense. Before deployment, you must replace all square-bracket placeholders with your specific model catalog, pricing, and budget constraints.

textCopy
You are a cost-aware routing classifier. Your task is to analyze the following user request and select the most appropriate model tier based on its complexity and our defined budget constraints.

### COMPLEXITY RUBRIC
Evaluate the request against these criteria:
- **Low Complexity:** Simple factual lookup, single-step instruction, well-defined format conversion, or common knowledge query. No multi-step reasoning required.
- **Medium Complexity:** Multi-step reasoning, comparison of a few entities, summarization of a single document, or generation with specific stylistic constraints.
- **High Complexity:** Multi-document synthesis, complex logical deduction, advanced code generation with multiple dependencies, or creative tasks requiring deep domain expertise.

### MODEL TIER CATALOG
- **tier: "lightweight"**
  model_id: [LIGHTWEIGHT_MODEL_ID]
  cost_per_1k_tokens: [LIGHTWEIGHT_COST]
  suitable_for: "Low Complexity"
- **tier: "standard"**
  model_id: [STANDARD_MODEL_ID]
  cost_per_1k_tokens: [STANDARD_COST]
  suitable_for: "Medium Complexity"
- **tier: "premium"**
  model_id: [PREMIUM_MODEL_ID]
  cost_per_1k_tokens: [PREMIUM_COST]
  suitable_for: "High Complexity"

### BUDGET CONSTRAINT
- **Hard Cost Cap:** The selected tier's cost must not exceed [MAX_COST_PER_1K_TOKENS].
- **Default Tier:** If no tier satisfies the constraints, default to the "lightweight" tier.

### USER REQUEST
[USER_INPUT]

### INSTRUCTIONS
1.  Assess the complexity of the user request using the rubric.
2.  Select the cheapest model tier that is suitable for the assessed complexity and does not violate the hard cost cap.
3.  Output your decision in the following JSON format. Do not include any other text.

### OUTPUT SCHEMA
{
  "complexity_assessment": "Low" | "Medium" | "High",
  "complexity_rationale": "A brief, one-sentence justification for the complexity score.",
  "selected_tier": "lightweight" | "standard" | "premium",
  "selected_model_id": "The model_id from the chosen tier.",
  "cost_justification": "A brief explanation confirming the selection is the cheapest suitable option within the budget."
}

To adapt this template, start by populating the MODEL TIER CATALOG with your actual model IDs and per-token costs. The COMPLEXITY RUBRIC should be tuned to match the observed performance boundaries of your specific models; you may need to adjust the definitions of 'Low,' 'Medium,' and 'High' based on your own evaluations. The OUTPUT_SCHEMA is designed for direct parsing by your application's routing logic. Ensure your router reads the selected_model_id field and dispatches the [USER_INPUT] to that model's API endpoint. The complexity_rationale and cost_justification fields are critical for logging and debugging routing decisions, allowing you to audit why a premium model was selected for a given request.

This prompt is designed to sit inside a model routing middleware layer—code that intercepts every user request, classifies its complexity, and dispatches it to the appropriate model tier. The prompt itself is stateless and should be treated as a pure function: it receives a request payload and a cost configuration, and it returns a tier selection with justification. The surrounding harness is responsible for enforcing the decision, not the prompt. This means the application layer must own the actual model dispatch, cost accounting, and any overrides based on user tier or system state.

Validation and retry logic are critical because a malformed tier selection can silently route a complex request to a lightweight model, producing a poor user experience. The harness should validate the output against a strict schema: tier must be one of the defined enum values (lightweight, standard, premium), confidence must be a float between 0.0 and 1.0, and justification must be a non-empty string. If validation fails, retry the prompt once with the validation error appended as a [CONSTRAINTS] update. If the retry also fails, log the failure and fall back to the standard tier as a safe default—never fall back to premium to avoid cost runaways, and never fall back to lightweight for unclassified requests. Logging should capture the input complexity signals, the selected tier, confidence score, justification, and any validation failures for later cost and quality analysis.

Model choice for the router itself matters. This prompt performs classification, not generation, so a fast, cheap model (e.g., GPT-4o-mini, Claude Haiku, or a fine-tuned small classifier) is usually sufficient. Avoid using your premium tier model to decide whether to use the premium tier—that defeats the cost-saving purpose. Tool use and RAG are not required for this prompt in its basic form, but if your complexity rubric references external knowledge (e.g., domain-specific complexity heuristics), inject that context via the [COMPLEXITY_RUBRIC] variable rather than relying on the model's training data. Human review is not needed per routing decision, but you should implement a periodic audit where a sample of routing decisions is reviewed for accuracy, especially for requests near complexity boundaries. Set up a dashboard tracking premium-tier utilization rate, lightweight-tier failure rate (requests that later required escalation), and average cost per request to detect routing drift.

Fallback and circuit breaker patterns protect against router failure. If the routing prompt consistently fails validation or times out, the harness should open a circuit and route all requests to the standard tier while alerting the on-call engineer. Do not let a broken router block user requests. Additionally, implement a cost cap enforcement layer outside the prompt: track cumulative spend per time window, and if the premium tier is being selected more frequently than budgeted, inject an updated [BUDGET_CONSTRAINT] into the prompt to tighten the cost threshold. The prompt should never be the sole guardrail on spend—the application layer must enforce hard limits.