Comparison Query Detection and Routing Prompt

This prompt is for search engineers and RAG orchestration developers who need to reliably detect when a user query expresses comparative intent—for example, 'compare AWS Lambda vs. Cloudflare Workers for latency' or 'what's the difference between PostgreSQL and MySQL for time-series data.' The core job-to-be-done is classification and routing: the prompt must distinguish comparison queries from factual lookups, procedural how-tos, summarization requests, and other query types, then route the comparison query to a structured comparison retrieval strategy rather than a standard QA pipeline. The ideal user is an engineer building a retrieval dispatch layer who already has multiple retrieval backends or query formulation strategies and needs a reliable classifier to prevent comparison queries from hitting a generic vector index that will return undifferentiated results.

Use this prompt when your system has a dedicated comparison retrieval path—one that can generate side-by-side attribute extraction, structured trade-off matrices, or multi-entity evidence gathering. The prompt expects you to provide a [QUERY] string and optionally [CONVERSATION_HISTORY] for multi-turn disambiguation. It returns a classification label and a routing decision. Do not use this prompt if your retrieval system has no comparison-specific strategy; routing to a path that doesn't exist will degrade the user experience. Do not use this prompt for single-entity analysis ('tell me about AWS Lambda') or for queries that only superficially mention two entities without requesting a comparison ('what is PostgreSQL and who uses it'). Those belong in factual or single-entity retrieval paths.

The prompt includes explicit negative examples and edge-case handling for queries that mention multiple entities but are not comparisons—such as listing queries ('list all AWS database services'), sequential questions ('first tell me about Redis, then about Memcached'), or co-occurrence without contrast ('how do Docker and Kubernetes work together'). These are common failure modes where a naive keyword-based classifier would misroute. The prompt also handles implicit comparisons ('which is faster', 'better for startups', 'cheaper option') where the user doesn't explicitly name both entities. Before deploying, run the eval checks described in the implementation harness section to measure missed-comparison and false-positive rates against your own query distribution. If your domain has high-stakes comparisons (medical treatments, financial products, legal strategies), add a human review step before the comparison retrieval results are surfaced to end users.

This section provides the core prompt template you will adapt and embed in your retrieval dispatch layer. The prompt is designed to sit between the user's natural language query and your retrieval strategy selector. Its job is to classify whether the query expresses a comparison intent—product vs. product, vendor vs. vendor, concept vs. concept—and, if so, to produce a structured routing decision that downstream orchestration code can act on. The template uses square-bracket placeholders for all variable inputs, making it safe to copy into your codebase, populate programmatically, and version alongside your application logic.

textCopy
You are a query intent classifier for a retrieval system. Your task is to analyze the user's query and determine whether it expresses a comparison intent. A comparison query asks to evaluate, contrast, or differentiate between two or more entities, products, vendors, concepts, approaches, or options.

[EXAMPLES]

Query: "[INPUT]"

[CONTEXT]

Classify the query into exactly one of these categories:
- COMPARISON: The user wants to compare, contrast, or differentiate between multiple items.
- NON_COMPARISON: The query is factual, procedural, definitional, summarization, or any other non-comparative intent.

If the query is COMPARISON, also extract the entities being compared and the comparison dimensions (features, pricing, performance, etc.) if explicitly stated or strongly implied.

Return your response as a valid JSON object matching this schema:
[OUTPUT_SCHEMA]

[CONSTRAINTS]

Do not include any text outside the JSON object.

To adapt this template for your system, start by populating [EXAMPLES] with 3–6 few-shot demonstrations that cover the comparison types your domain sees most often—product comparisons, vendor evaluations, technical trade-offs, and edge cases like implicit comparisons ("Is X better?"). The [CONTEXT] placeholder should be filled with any available session history, user role, or domain metadata that helps disambiguate intent. The [OUTPUT_SCHEMA] field must contain your exact JSON schema definition, including required fields like intent, confidence, entities, and dimensions. The [CONSTRAINTS] placeholder is where you add domain-specific rules: for example, "If only one entity is mentioned but the query implies comparison (e.g., 'better than alternatives'), classify as COMPARISON and mark entities as incomplete." After adapting the template, wire the JSON output into your routing logic so that COMPARISON queries trigger your structured comparison retrieval pipeline—typically a multi-entity search with side-by-side evidence aggregation—while NON_COMPARISON queries follow your standard QA or factual retrieval path. Before shipping, run this prompt against a golden eval set that includes both obvious comparisons and tricky non-comparison queries that use comparative language ("What's the fastest database?") to measure false-positive and false-negative rates.

The Comparison Query Detection and Routing Prompt is designed to sit at the entry point of a retrieval dispatch layer, before any query rewriting or index selection occurs. Its job is narrow: classify whether the user's input expresses comparative intent (product vs. product, vendor vs. vendor, concept vs. concept) and, if so, route the query to a structured comparison retrieval strategy rather than standard QA or factual lookup. In a production harness, this prompt should be called as a lightweight pre-retrieval classifier with a strict timeout and a deterministic fallback behavior. The output is a structured classification decision, not an answer, so the integration point is typically a gateway function that inspects the classification result and branches the retrieval pipeline accordingly.

Wire the prompt into your application as a synchronous pre-retrieval step. The caller should pass the raw user query as [INPUT] and optionally include [SESSION_CONTEXT] if prior turns exist. The model should return a JSON object with at minimum a query_type field (expected values: comparison, non_comparison) and a confidence score between 0.0 and 1.0. Implement a validation layer that rejects malformed JSON, missing required fields, or confidence scores outside the valid range. If validation fails, retry once with the same input and a stricter output format instruction appended. If the retry also fails, log the failure and route to the default non-comparison retrieval path—never block the user waiting for classification. For model choice, use a fast, low-cost model (e.g., GPT-4o-mini, Claude Haiku, or a fine-tuned small classifier) since this is a narrow classification task, not a generation task. Latency budget should be under 500ms including network overhead.

Add structured logging at the classification decision point: capture the query, the classified type, the confidence score, the model used, latency, and whether the classification triggered a comparison retrieval path or default path. This log becomes your primary observability surface for tuning thresholds and detecting drift. Set a confidence threshold (start at 0.7) below which you treat the classification as uncertain and route to the default path. Monitor two failure modes closely: false positives (non-comparison queries routed to comparison retrieval, producing irrelevant structured comparisons) and false negatives (comparison queries hitting standard QA, producing flat answers that miss the comparative structure users expect). Build an eval set of at least 50 labeled queries covering explicit comparisons ('compare X and Y'), implicit comparisons ('which is better, X or Y?'), and non-comparison queries that contain entity pairs but no comparative intent ('tell me about X and Y'). Run this eval set weekly and whenever you update the prompt or model. If the domain involves regulated content (healthcare, finance, legal), add a human review step for any comparison query that would surface in a customer-facing answer without source attribution.