Model Suitability Decision Prompt for AI Routing

This prompt is designed for AI platform and gateway teams who need to automate the selection of the most appropriate model for a given task from a constrained, internal catalog. The core job-to-be-done is to programmatically route an incoming request—along with its specific requirements for cost, latency, capability, and accuracy—to the best-fit model endpoint. The ideal user is a routing engine or an orchestration layer, not an end-user chatting with a model. This prompt acts as the decision function within a model gateway, consuming a structured task description and outputting a strict, machine-readable routing decision.

Use this prompt when you have a predefined set of available models, each with a known profile of strengths, weaknesses, costs, and latency characteristics. It is appropriate when the routing logic is too complex for simple if/else rules but must remain fully deterministic and auditable. The prompt requires a well-structured input that includes the task's functional requirements, a non-functional requirements block (cost budget, latency ceiling), and the full model catalog with capabilities. It is not a general-purpose recommendation engine. Do not use this prompt to compare models for a one-off research task or to generate a human-readable report; its output is a single, constrained decision object meant for machine consumption.

Before implementing, define the failure modes for your system. The prompt must include a fallback model selection strategy for when no model perfectly meets the criteria. The output schema should enforce a mandatory fallback_model field and a rationale that justifies the trade-off. You must pair this prompt with a validation layer that rejects any output containing a model ID not present in the provided catalog. In high-stakes environments where a routing error could cause a compliance issue or a significant cost overrun, log the full routing decision and rationale for audit, and consider a human-in-the-loop review step for requests that fall below a configurable confidence threshold.

This prompt template is designed to be dropped directly into an AI routing gateway or model selection layer. It forces the model to reason over a predefined catalog of available models, match task requirements against model capabilities, and output a structured decision with explicit justification. The template uses square-bracket placeholders that your application must populate at runtime: the task description, the available model catalog with capabilities and cost profiles, the evaluation criteria weights, and any hard constraints like maximum latency or budget caps.

textCopy
You are a model routing decision engine. Your job is to select the most appropriate model from the [MODEL_CATALOG] for the given task, applying the evaluation criteria and constraints specified below.

## Available Models
[MODEL_CATALOG]
<!-- Format each model as:
- model_id: string
- provider: string
- capabilities: list of supported task types
- context_window: integer (tokens)
- cost_per_1k_input_tokens: float
- cost_per_1k_output_tokens: float
- typical_latency_ms: integer (p50)
- max_output_tokens: integer
- supports_tools: boolean
- supports_structured_output: boolean
- availability_tier: "ga" | "beta" | "deprecated"
- recommended_for: list of task categories
-->

## Task Description
[TASK_DESCRIPTION]
<!-- Describe the task in detail: input type, required output format, complexity, domain, and any special requirements like tool use, structured output, or multimodal inputs. -->

## Evaluation Criteria
[EVALUATION_CRITERIA]
<!-- Specify weighted criteria, for example:
- capability_match: weight 0.4
- cost_efficiency: weight 0.25
- latency_requirement: weight 0.2
- reliability_and_availability: weight 0.15
-->

## Constraints
[CONSTRAINTS]
<!-- Hard constraints that must be satisfied:
- max_latency_ms: integer or "none"
- max_cost_per_call: float or "none"
- required_capabilities: list of mandatory capabilities
- min_context_window: integer or "none"
- require_structured_output: boolean
- require_tool_support: boolean
- exclude_providers: list of providers to exclude
- availability_tier_minimum: "ga" | "beta"
-->

## Output Schema
Return a JSON object with this exact structure:
{
  "selected_model": {
    "model_id": "string",
    "provider": "string",
    "confidence": 0.0-1.0,
    "rationale": "string explaining why this model was selected over alternatives"
  },
  "fallback_model": {
    "model_id": "string",
    "provider": "string",
    "trigger_condition": "string describing when to fall back"
  },
  "alternatives_considered": [
    {
      "model_id": "string",
      "rejection_reason": "string explaining why this model was not selected"
    }
  ],
  "risk_flags": [
    "string describing any risks or concerns with the selection"
  ],
  "estimated_cost_per_call": float,
  "estimated_latency_ms": integer
}

## Decision Rules
1. Eliminate any model that violates a hard constraint before scoring.
2. Score remaining models against the weighted evaluation criteria.
3. If no model satisfies all constraints, return the closest match with risk_flags explaining the gap.
4. Always provide a fallback model from a different provider when available.
5. If multiple models score within 5% of each other, prefer the lower-cost option.
6. Never select a deprecated model unless no alternative exists.

## Examples
[EXAMPLES]
<!-- Provide 2-3 example inputs and expected outputs showing correct decision-making across different scenarios: cost-sensitive, latency-sensitive, capability-constrained. -->

Now evaluate the task and return your decision.

To adapt this template, start by defining your [MODEL_CATALOG] as a structured data object your application can serialize into the prompt. Keep the catalog current—stale model data produces routing decisions that waste money or break latency SLAs. The [EVALUATION_CRITERIA] weights should reflect your actual production priorities; if you say latency matters but weight cost higher, the model will optimize for cost and you'll get slow responses. Populate [EXAMPLES] with real decisions your team would make manually, including edge cases where no model is perfect. The output schema includes risk_flags and fallback_model fields because routing decisions fail in production—plan for degradation before it happens.

Before deploying this prompt into a routing gateway, validate that your application layer can parse the JSON output and act on the fallback_model field when the primary model returns errors or exceeds latency budgets. Log every routing decision with the full prompt context and model response for debugging. If your routing decisions affect regulated workflows, add a human review step for low-confidence selections where confidence falls below your threshold. Test the prompt against your catalog weekly as models deprecate and new ones launch—a routing prompt that recommends a retired model is worse than no routing at all.

This prompt is designed to be the decision layer inside an AI gateway or model router. It should not be called directly by end users. Instead, wrap it in an application service that receives an incoming task specification, enriches it with current capability and cost data from your model catalog, and then calls the prompt to produce a structured routing decision. The output of this prompt is a JSON object containing a primary model selection, a fallback model, and a justification block. Your application must parse this JSON, validate it against the expected schema, and then use the primary_model_id to dispatch the actual inference request to the correct model endpoint.

Before calling the prompt, assemble the [MODEL_CATALOG] input by querying your live model registry. This catalog must include each model's model_id, capabilities (as a list of supported task types), cost_per_1k_tokens, avg_latency_ms, context_window, and availability_status. Do not hardcode this list; it must reflect current deployment state. The [TASK_REQUIREMENTS] input should be derived from the incoming user or system request and must specify task_type (from a controlled vocabulary matching your catalog's capability tags), priority (latency, cost, or quality), max_budget_tokens, and any required_capabilities. The [CONSTRAINTS] block should include your organization's routing policies, such as "prefer_private_deployment": true or "max_cost_per_1k_tokens": 0.02. After the prompt returns, validate the primary_model_id and fallback_model_id against your catalog to prevent hallucinated model identifiers. If validation fails, log the invalid output and retry once with a stricter prompt that includes the validation error message. If the retry also fails, escalate to a human operator and route to a safe default model.

Implement a lightweight evaluation loop around this prompt. For every routing decision, log the input task requirements, the selected model, the justification, and the actual outcome metrics (latency, cost, success/failure) after the task completes. Periodically run a golden set of routing scenarios through the prompt and compare the decisions against expected selections. Flag any drift where the prompt starts over-prioritizing cost at the expense of capability or vice versa. For high-stakes routing decisions—such as those involving regulated data or customer-facing interactions—require a human approval step before the routing decision is executed. The approval UI should display the task summary, the prompt's recommended model, the justification, and a one-click override to the fallback model. Finally, treat the prompt template itself as a versioned artifact in your CI/CD pipeline. Any change to the model catalog schema, the task type vocabulary, or the routing policy should trigger a re-evaluation of this prompt against your golden test set before deployment.