Multi-Hop Retrieval Necessity Detection Prompt

This prompt operates as a binary classifier at the entry point of a retrieval dispatch layer. Its sole job is to inspect a user query and decide whether answering it requires a single round of retrieval or a planned sequence of chained retrieval steps. You should use this prompt when you operate a routing layer that can branch between a standard RAG pipeline and a multi-hop agent loop. The classification result determines which downstream execution path the query takes, making this a critical gating decision that affects both latency and answer completeness.

Deploy this prompt when your system has two distinct retrieval execution paths: a fast single-pass RAG pipeline for simple lookups and a slower multi-hop agent loop for complex reasoning. The prompt prevents two costly failure modes. First, it avoids wasting compute and latency budget on multi-hop orchestration for queries that a single retrieval round can answer correctly. Second, it prevents single-pass retrieval from silently failing on questions that demand intermediate evidence—for example, 'What was the revenue of the company that acquired our biggest competitor last year?' requires first identifying the competitor, then finding the acquirer, then looking up that acquirer's revenue. Without necessity detection, a single-pass system would retrieve irrelevant financial data for the wrong entity.

Do not use this prompt when every query already flows through a fixed multi-step agent architecture. If your system always performs planning, decomposition, and iterative retrieval regardless of query complexity, adding a necessity detection step introduces latency without changing behavior. Similarly, avoid this prompt in systems where retrieval is always single-pass and there is no multi-hop execution path to route to—the classification has no downstream effect. The prompt is also unnecessary for latency-critical applications where the multi-hop path is too slow to be viable for any query, making the routing decision moot.

Before deploying, ensure your routing infrastructure can act on the classification output. The prompt returns a structured decision with a confidence score, but that decision must connect to actual execution logic: a single-pass RAG pipeline on one branch and a multi-hop agent with planning, tool-use, and iterative retrieval on the other. Log every classification decision alongside the query and final answer quality metrics. This logging creates a feedback loop that reveals whether the classifier is routing queries correctly or whether threshold adjustments are needed. Common failure patterns include classifying genuinely multi-hop questions as single-pass (causing incomplete answers) and routing simple lookups through the expensive multi-hop path (wasting latency and compute).

This template is designed to be dropped directly into your retrieval orchestration layer. It forces the model to make a binary decision—single-hop or multi-hop—and to justify that decision with explicit reasoning about hidden dependencies, entity relationships, and temporal or logical prerequisites. The output is structured JSON so your router can parse the decision programmatically without fragile string matching.

textCopy
You are a retrieval strategy classifier. Your job is to analyze a user query and determine whether answering it requires multiple sequential retrieval steps (multi-hop) or can be satisfied with a single retrieval round (single-hop).

A query requires multi-hop retrieval when:
- Answering it requires information that is not directly stated in any single document but must be assembled from two or more separate pieces of evidence.
- It asks about a relationship between two entities where the connection is not explicitly documented in one place.
- It contains a temporal dependency where you must first find an event and then find information about that event's consequences or context.
- It asks for a property of an entity that must first be identified through another query (e.g., "What was the population of the city where the 2024 summit was held?").

A query is single-hop when:
- The answer is likely to exist in a single document or passage.
- It asks for a direct fact, definition, or summary of a known topic.
- Any apparent dependencies can be resolved by a well-scoped single retrieval with good recall.

Analyze the query below and return a JSON object with the following schema:
{
  "requires_multi_hop": boolean,
  "confidence": "high" | "medium" | "low",
  "reasoning": "A concise explanation of what dependencies exist or why none were found.",
  "suggested_hop_sequence": ["First retrieval step description", "Second retrieval step description"] | null
}

If requires_multi_hop is false, suggested_hop_sequence must be null.

[QUERY]
[CONSTRAINTS]

To adapt this template, replace [QUERY] with the user's raw input and [CONSTRAINTS] with any domain-specific rules—for example, "Treat any query mentioning 'compare' or 'difference between' as single-hop if a comparison document likely exists." If your system uses a fixed set of retrieval backends, you can extend the output schema with a recommended_strategy field that maps to your internal routing enum. Always validate the output JSON before acting on the routing decision; if the model returns malformed JSON or a confidence of "low," route to a conservative fallback that attempts single-hop retrieval first and escalates to a human or a planning agent if the answer quality is insufficient.

This prompt acts as a pre-retrieval classifier in your query pipeline. It should be invoked after query normalization but before any index call. The model receives the user's query and optional conversation context, then returns a structured decision: multi_hop_required (boolean), reasoning (string), and a confidence score. This output gates whether the request proceeds to a single-pass RAG flow or a multi-hop agent loop. Wire it as a synchronous step with a strict timeout—if the classifier takes longer than 2 seconds, default to single-pass retrieval and log the timeout for latency analysis.

Validation and retry logic: Parse the model's output against a strict schema. If multi_hop_required is not a boolean or confidence falls outside 0.0–1.0, retry once with the same input plus a repair instruction. If the second attempt also fails validation, log the raw output, set multi_hop_required to false, and route to single-pass retrieval. This conservative fallback prevents a broken classifier from blocking all traffic. For high-stakes domains (legal, medical, financial), flag low-confidence predictions (confidence < 0.7) for human review or route to a more capable model for re-classification before committing to an expensive multi-hop plan.

Model choice and tool integration: Use a fast, instruction-tuned model (e.g., GPT-4o-mini, Claude Haiku, or a fine-tuned local classifier) for this detection step. The prompt does not require tool calls or retrieval—it's a pure reasoning task. However, the output should be consumed by a downstream router that either invokes a single-shot RAG pipeline or triggers a multi-hop agent with access to search tools, sub-question decomposition prompts, and a state tracker. Log every classification decision with the query hash, model version, confidence score, and chosen path. This trace data is essential for calibrating the confidence threshold and identifying query patterns where the classifier consistently fails.

What to avoid: Do not use this prompt as the sole gatekeeper without a timeout and fallback. Do not skip validation on the structured output—unchecked model drift will silently break your routing logic. Do not run multi-hop retrieval for every query flagged as multi-hop; pair this classifier with a cost budget and a maximum hop limit (typically 3–5 steps) to prevent runaway agent loops. Finally, monitor the ratio of multi-hop to single-hop decisions in production. A sudden spike often indicates prompt drift, a model update, or a change in user query patterns that requires recalibration.