Model-Selection Intent Prompt for Router Systems

This prompt is for infrastructure engineers building model routers that must select the optimal model per request. The job-to-be-done is not simple intent classification; it's a reasoned dispatch decision that weighs a user's unstructured input against a defined pool of models, each with different cost, latency, and capability profiles. The ideal user is an AI platform engineer who already has a model pool with defined capabilities and needs the router to make a trade-off decision, not just a keyword match. You should use this prompt when the routing logic requires understanding the complexity, domain, and constraints of a request to map it to the best-fit model.

Do not use this prompt for simple intent classification that maps to a single downstream workflow, like a basic support ticket router that sends all 'billing' questions to one queue. This prompt is overkill if your system dispatches work to a single model or if the routing decision is a static lookup. It assumes you have a multi-model architecture and that the routing decision requires reasoning about trade-offs. Before implementing, you must have a well-defined model pool with documented capabilities, cost profiles, and latency characteristics. The prompt's value is in its ability to articulate why a specific model was chosen, providing a routing rationale that is auditable and debuggable.

To get started, define your model pool as a structured input to the prompt, including each model's strengths, weaknesses, and constraints. Then, integrate this prompt into your request pipeline before model invocation. The next section provides the copy-ready template. After adapting it, you'll need to build a validation harness to ensure the selected model is actually in your pool and that the rationale is coherent. For high-stakes routing, always log the full prompt and response for offline evaluation and A/B testing against alternative routing strategies.

This template is the core instruction set for a model router. It takes a user request, a catalog of available models with their capabilities and cost profiles, and a set of operational constraints, then outputs a structured routing decision. The prompt is designed to be injected into a system prompt for a fast, inexpensive classifier model (the 'router') that sits in front of a pool of more powerful, specialized models. The square-bracket placeholders must be populated by your application logic before the prompt is sent to the router model.

textCopy
You are a model-selection router. Your task is to analyze the user's request and select the most appropriate model from the available catalog to handle it. You must balance capability requirements, cost, and latency.

## AVAILABLE MODEL CATALOG
[MODEL_CATALOG]

## OPERATIONAL CONSTRAINTS
- Maximum cost budget per request: [COST_BUDGET]
- Maximum latency target (ms): [LATENCY_BUDGET]
- Default model for out-of-scope requests: [DEFAULT_MODEL]

## USER REQUEST
[USER_INPUT]

## INSTRUCTIONS
1. Analyze the user's request to determine the required capabilities (e.g., reasoning complexity, domain expertise, tool use, multimodality).
2. Select the single best model from the catalog that meets all requirements and stays within the operational constraints.
3. If no model is suitable, or the request is out of scope, select the default model and set the `abstain` flag to `true`.
4. Provide a concise, evidence-based rationale for your selection.

## OUTPUT SCHEMA
You must respond with a single, valid JSON object conforming to this schema:
{
  "model_id": "string",
  "rationale": "string",
  "abstain": false,
  "estimated_cost": "number",
  "estimated_latency_ms": "number"
}

To adapt this template, replace [MODEL_CATALOG] with a structured list of your available models. This list should be a clear, text-based representation of a JSON array, including fields like id, description, strengths, cost_per_1k_tokens, and avg_latency_ms. The [COST_BUDGET] and [LATENCY_BUDGET] placeholders should be replaced with numeric values representing your application's hard limits. The [USER_INPUT] is the raw, unmodified text from the end-user. Before deploying, validate that the router model reliably produces JSON conforming to the output schema. Implement a post-processing step in your harness to parse the JSON, check the abstain flag, and route the [USER_INPUT] to the model specified by model_id. If the router's output fails schema validation, your harness should fall back to the [DEFAULT_MODEL] and log the failure for debugging.

This prompt is designed to be the decision layer inside a model router, not a standalone chatbot. The application harness must receive the structured JSON output, validate it against the available model pool, enforce cost and latency budgets, and then dispatch the request to the selected model. The prompt itself does not execute the routing; it produces a routing recommendation that your application code must verify and act on. Treat the prompt output as an advisory decision that requires runtime validation before any model invocation occurs.

The implementation flow should follow a strict sequence: (1) assemble the prompt with the user's input, the available model catalog including capabilities and cost profiles, and the current latency budget; (2) call a fast, low-cost model to execute this classification prompt; (3) parse the JSON output and validate that the model_identifier exists in the active model pool and is currently healthy; (4) check that the estimated cost and latency from the routing rationale do not exceed the configured budget; (5) if validation passes, dispatch the original user input to the selected model; (6) if validation fails, fall back to a default model or escalate to a human operator. Log every routing decision with the full prompt output, validation result, and fallback reason for later analysis.

For production deployment, implement a retry policy on the classification step itself: if the classification model returns malformed JSON or a model identifier that fails validation, retry once with a stricter output format instruction before falling back. Add a circuit breaker that routes all traffic to a default model if the classification step exceeds a latency threshold (e.g., 200ms) or error rate (e.g., 5% over a rolling window). Instrument the harness to emit metrics on classification accuracy, routing latency, cost-per-request, and fallback rate. Run A/B comparisons by splitting traffic between the routed model selection and a baseline model to measure whether the router actually improves quality, cost, or latency relative to a static default. The routing rationale field is your primary debugging artifact when a routing decision looks wrong in production traces.