Query Complexity Scoring Prompt for Strategy Routing

This prompt is designed for retrieval pipeline engineers who need a deterministic, auditable method to classify a user query by its complexity before selecting a retrieval strategy. The core job-to-be-done is to score a query on dimensions such as multi-hop requirements, entity density, and constraint count, producing a structured complexity profile. This profile acts as a signal for a routing layer, enabling it to dispatch the query to the most appropriate backend—whether that's a simple keyword search, standard vector retrieval, a hybrid approach, or a full multi-hop agent workflow. The ideal user is an orchestration engineer or RAG architect who already manages multiple retrieval backends and needs to move beyond hardcoded rules or naive LLM routing to a more transparent, tunable dispatch mechanism.

You should use this prompt when your system has a clear cost and latency gradient across retrieval strategies, and you need to reserve expensive, advanced strategies for the subset of queries that genuinely require them. It is particularly effective when you have already established distinct retrieval paths (e.g., a fast keyword index, a dense vector store, and a slower agent-driven multi-hop process) and need a scoring layer to bridge the gap between the user's raw query and the backend selection logic. The prompt's structured output allows you to set explicit thresholds: for example, a query scoring below a certain multi-hop threshold might be routed directly to vector search, while a query with a high constraint count and entity density might trigger a hybrid retrieval with metadata filtering. This approach gives you fine-grained control over your retrieval budget and makes the routing logic auditable, as you can log the complexity scores alongside the final routing decision.

Do not use this prompt as a standalone answer generator or for queries that are already pre-classified by a hardcoded rule engine. If your system has a simple, static set of rules (e.g., 'all queries from this endpoint go to vector search'), adding a complexity scoring layer introduces unnecessary latency and cost. Similarly, avoid using this prompt when you have only a single retrieval backend; the scoring output provides no actionable routing value in a homogeneous retrieval environment. Finally, this prompt is not a substitute for a full query decomposition or planning step—it scores complexity but does not generate the sub-questions or execution plan required for a multi-hop agent. If the score indicates high complexity, you should hand off to a dedicated planning prompt or agent workflow rather than expecting this prompt to produce the final answer.

This prompt template is designed to be dropped into a retrieval orchestration layer. It takes a user query and a set of defined complexity dimensions, and returns a structured JSON object with a score for each dimension and a final complexity level. The output is intended to be consumed by a routing function that selects a retrieval strategy—such as single-pass vector search for simple queries or multi-hop agentic retrieval for complex ones.

textCopy
System: You are a query complexity analyzer for a retrieval system. Your job is to score a user query on multiple dimensions of complexity to determine the appropriate retrieval strategy. Return only valid JSON that matches the [OUTPUT_SCHEMA]. Do not include any other text.

Complexity Dimensions:
- multi_hop_required (0-3): Does the query require chaining multiple pieces of information? 0 = single fact, 3 = requires 3+ dependent retrieval steps.
- entity_density (0-3): How many distinct entities, products, or named concepts are referenced? 0 = none, 3 = 5+ distinct entities.
- constraint_count (0-3): How many explicit filters, conditions, or constraints are present? 0 = none, 3 = 4+ constraints.
- temporal_complexity (0-3): Does the query involve relative time, date ranges, or versioning? 0 = no time aspect, 3 = multiple time windows or complex relative expressions.
- comparison_required (0-3): Does the query ask for a comparison or trade-off analysis? 0 = no comparison, 3 = multi-entity, multi-attribute comparison.
- domain_specificity (0-3): How specialized is the vocabulary or domain knowledge required? 0 = general language, 3 = requires deep domain terminology and concepts.

Scoring Guidelines:
- Score each dimension independently.
- Use the full 0-3 range; do not default to middle values.
- A score of 0 means the dimension is absent.
- A score of 3 means the dimension is extreme for that category.

Overall Complexity Level:
- "simple": Sum of all scores <= 4. Route to single-pass vector search.
- "moderate": Sum of all scores 5-10. Route to hybrid retrieval with re-ranking.
- "complex": Sum of all scores >= 11. Route to multi-hop agentic retrieval with decomposition.

User Query: [USER_QUERY]

[OUTPUT_SCHEMA]

To adapt this template, replace [USER_QUERY] with the raw user input and [OUTPUT_SCHEMA] with your application's expected JSON structure. A typical schema includes integer fields for each dimension, an integer for total_score, and a string for complexity_level. If your routing logic uses different thresholds, adjust the scoring guidelines in the system prompt. For high-stakes applications where misrouting could cause compliance issues, add a [CONSTRAINTS] block specifying that the model should flag queries it cannot confidently score, and route those to a human review queue.

Before deploying, test this prompt against a golden dataset of 50-100 queries that span the full complexity range. Validate that the output is parseable JSON in at least 98% of cases and that the complexity_level assignment matches your expected routing decision in at least 90% of cases. Common failure modes include the model over-scoring domain_specificity for queries with a single technical term, or under-scoring multi_hop_required when the dependency is implicit. Add a post-processing validator that checks for missing fields, out-of-range scores, and schema compliance before the routing decision is made. For production, log the raw scores alongside the final route taken so you can recalibrate thresholds as your retrieval system evolves.

This prompt is designed to sit at the entry point of a retrieval dispatch layer, classifying every incoming user query before it hits any index. The implementation harness should treat the prompt as a synchronous, stateless function that receives a raw query string and returns a structured complexity score. Because the output directly controls routing decisions—simple queries go to a fast, cheap vector index while complex queries trigger multi-hop decomposition or hybrid retrieval—the harness must enforce strict output validation before any downstream system consumes the score. A malformed or hallucinated score can silently route a complex query to a simple index, producing an incomplete answer that erodes user trust without an obvious error signal.

Wire the prompt into your application as a pre-retrieval gate. The caller should pass the raw user query as [INPUT] and optionally include conversation context as [CONTEXT] if the query is part of a multi-turn session. The model should return a JSON object with integer scores for each dimension (multi-hop requirements, entity density, constraint count, temporal complexity, and ambiguity) plus a final composite score. Implement a JSON schema validator in your harness that rejects any response missing required fields, containing out-of-range values, or failing to parse. On validation failure, retry once with a stricter prompt that includes the validation error message. If the second attempt also fails, log the failure, route the query to a conservative high-capability retrieval path, and alert the on-call channel. For model selection, use a fast, instruction-following model such as Claude 3.5 Haiku or GPT-4o-mini to keep latency under 200ms at the dispatch gate. Avoid using a slow reasoning model here because the scoring step adds overhead to every query, and the cost of delay compounds across your user base.

Calibrate your routing thresholds against production data, not intuition. After deploying the harness, log the raw query, the scored dimensions, and the selected retrieval strategy for at least a week before tuning thresholds. Sample queries that received borderline scores and have a human reviewer label whether the routing decision was correct. Use this labeled set to adjust your threshold values and to measure precision and recall of the complexity classifier. Build a dashboard that tracks score distributions, validation failure rates, and routing decision changes over time. The most common production failure mode is threshold drift: as your user base or content corpus changes, the distribution of query complexity shifts, and previously reasonable thresholds begin misrouting queries. Schedule a monthly review of threshold calibration against recent query samples to catch this drift before it degrades answer quality.