Retrieval Strategy Dispatch Prompt Template

This prompt is for orchestration engineers building retrieval routing layers in RAG systems. Its job is to act as a dispatcher: it receives a raw user query, classifies its intent and complexity, and outputs a structured decision that maps the query to the correct retrieval strategy and backend combination. Use this prompt when your system has multiple retrieval paths (dense vector, sparse keyword, graph, hybrid) and you need a single classification step to avoid sending every query to every backend. The ideal user is an infrastructure or platform engineer who already understands the available retrieval backends and their performance characteristics, and who needs a deterministic, auditable routing decision before query execution.

This prompt assumes you have already defined your available strategies and backends and can pass them as context. You must provide a catalog of retrieval strategies, each with a name, description, supported backends, and typical query patterns they handle. The prompt works best when the strategy catalog is stable and well-documented, because the model's classification accuracy depends on clear strategy definitions. Do not use this prompt when you have only one retrieval backend or when the retrieval strategy is static—a simple if-else rule in application code is cheaper, faster, and more reliable. Do not use this prompt for real-time latency-sensitive paths where sub-10ms decisions are required; model inference adds latency that may violate your SLO. Do not use this prompt when the query itself contains the routing signal in a structured field (e.g., a search_type parameter from the UI), because deterministic routing is always preferred over model-based classification.

Before deploying this prompt, you must define evaluation criteria for classification accuracy. Build a golden dataset of at least 100 queries with human-labeled strategy assignments, and measure whether the prompt's output matches the expected strategy and backend combination. Common failure modes include over-selecting the hybrid strategy for simple keyword queries, misclassifying comparative questions as factoid lookups, and failing to recognize multi-hop queries that require graph traversal. Plan for a human review step when the model's confidence is low or when the query falls into an undefined strategy gap. The next section provides the copy-ready prompt template you can adapt with your own strategy catalog and output schema.

This template is the core dispatch instruction for your retrieval routing layer. It forces the model to analyze the user's query for complexity, intent, and required capabilities before selecting a strategy. You must replace the square-bracket placeholders with your system's specific retrieval strategies, backend definitions, and routing rules. The template is designed to be stateless and can be called on every user turn before any retrieval is executed.

textCopy
You are a retrieval strategy router. Your job is to analyze a user query and select the most appropriate retrieval strategy and backend combination.

## Available Strategies
[STRATEGY_LIST]

## Available Backends
[BACKEND_LIST]

## Routing Rules
[ROUTING_RULES]

## Input
User Query: [USER_QUERY]
Conversation History (last 3 turns): [CONVERSATION_HISTORY]

## Task
1. Classify the query's primary intent and complexity.
2. Select exactly one strategy from the available list.
3. Select one or more backends required to execute that strategy.
4. Provide a brief justification for your selection.
5. If the query is ambiguous, select the strategy that requests clarification instead of guessing.

## Output Format
Return a valid JSON object with this exact schema:
{
  "classification": {
    "intent": "string",
    "complexity": "simple|moderate|complex",
    "is_ambiguous": boolean
  },
  "selected_strategy": "string",
  "selected_backends": ["string"],
  "justification": "string",
  "requires_clarification": boolean
}

## Constraints
- Do not execute retrieval. Only classify and route.
- If the query contains multiple intents, select the strategy that handles the dominant intent.
- For queries requiring real-time data, prefer backends marked as 'live'.
- For queries with explicit time constraints, include the temporal backend if available.
- If no strategy clearly matches, default to [FALLBACK_STRATEGY].

To adapt this template, start by defining your [STRATEGY_LIST] with clear, mutually exclusive options such as factoid_qa, summarization, comparison, multi_hop, temporal, or clarification_needed. Each strategy should map to a known retrieval pipeline. Your [BACKEND_LIST] should enumerate the actual indexes available, like dense_vector, sparse_keyword, graph_traversal, or hybrid. The [ROUTING_RULES] section is where you encode business logic, such as 'route all compliance queries to the policy backend' or 'queries shorter than 5 words are ambiguous by default.' After copying the template, immediately write a validation function that checks the output JSON against your actual strategy and backend enums to prevent hallucinated routing targets from reaching production.

The dispatch prompt is not a standalone component; it is the decision layer in a retrieval routing architecture. In production, this prompt sits between the user input and the retrieval backends. Its output—a strategy label and a set of backend assignments—must be parsed, validated, and used to invoke the correct downstream query rewriters and indexes. The harness should treat the LLM's classification as a control signal, not as the final answer. This means the application layer must enforce the dispatch decision, not trust it blindly.

Wire the prompt into a dispatch handler that receives the raw user query and any available session context. The handler constructs the prompt with the [USER_QUERY], [AVAILABLE_BACKENDS], and [SESSION_CONTEXT] placeholders populated from your retrieval infrastructure's capability registry and the current conversation state. After the LLM returns a classification, validate the output against a known schema: the strategy field must match one of your enumerated strategy types (e.g., factual_lookup, comparative_analysis, procedural_retrieval), and each entry in backends must reference a backend ID that exists in your live configuration. Reject any output that references unknown backends or strategies. On validation failure, retry once with the error injected into the prompt as [PREVIOUS_ERROR]. If the retry also fails, fall back to a safe default strategy (typically hybrid dense-plus-sparse) and log the incident for review.

Logging and observability are critical because dispatch errors cascade into every downstream retrieval step. Log the raw user query, the dispatch prompt version, the model's full classification output, the validation result, any retry attempts, and the final strategy selected. Tag each log entry with a trace ID that follows through to the retrieval backends and the final answer generation. This lets you measure dispatch accuracy by comparing the selected strategy against retrieval quality metrics (recall, precision, answer faithfulness) in your eval pipeline. If you observe that certain query types are consistently misclassified, use those logs to build few-shot examples or fine-tuning data for the dispatch prompt.

Model choice matters for latency and cost. The dispatch prompt is a classification task with a constrained output schema. In most cases, a fast, smaller model (such as GPT-4o-mini, Claude Haiku, or a fine-tuned open-weight model) is sufficient and keeps dispatch latency under 200ms. Reserve larger models for cases where the dispatch prompt must reason about ambiguous or multi-intent queries. Implement a circuit breaker that monitors dispatch latency and error rates; if the dispatch model exceeds your latency budget or error threshold, cut over to a static routing rule (e.g., always use hybrid retrieval) until the model endpoint recovers. Never let a dispatch failure block the user from getting results.

Before shipping, build a golden eval set of at least 200 queries covering each strategy type, edge cases (ambiguous queries, multi-intent questions, out-of-domain inputs), and failure modes you expect in production. For each query, define the expected strategy and acceptable backend assignments. Run this eval set against every prompt change and model upgrade. Track classification accuracy per strategy type, not just overall accuracy—a prompt that scores 95% overall but misclassifies all procedural queries is not ready for production. If your retrieval pipeline serves regulated or high-risk domains, add a human review step for dispatch decisions where the model confidence is below a threshold or where the selected strategy differs from the previous version's choice.