Multi-Hop Query Dependency Detection Prompt

This prompt serves as a routing decision point for RAG architects and agent developers who operate systems with both fast single-hop and slower multi-hop retrieval paths. Its job is to analyze a user question and determine whether the information needed to answer it can be gathered in one retrieval call or if the question contains implicit dependencies that require a sequence of queries where each subsequent step depends on results from prior steps. The output is a structured verdict—single-hop or multi-hop—along with a confidence score and a human-readable rationale. Your application can use this verdict to branch execution: send single-hop questions directly to a standard RAG pipeline for low-latency answers, and route multi-hop questions to a decomposition and chain-execution engine that will plan and execute dependent retrieval steps.

The ideal user is a retrieval pipeline engineer who has observed that some complex questions produce poor answers from a flat RAG setup and needs to avoid wasting latency budget on unnecessary chain execution for simple questions. Required context includes the full user question text and, optionally, conversation history if the question is a follow-up turn. Do not use this prompt when your system architecture mandates a fixed pipeline for all queries regardless of complexity, or when your retrieval backend cannot execute dependent query sequences—in those cases, the classification adds latency without actionable routing value. Also avoid this prompt for questions that are purely conversational, chitchat, or meta-instructions about the system itself; those should be caught by an upstream intent classifier before reaching this gate.

Before deploying, calibrate the dependency detection threshold against a labeled dataset of known single-hop and multi-hop questions from your domain. The most common production failure mode is false-positive chain triggers—questions that the model classifies as multi-hop but could have been answered correctly from a single well-constructed retrieval call. Each unnecessary chain execution adds latency and token cost. Start by setting a high confidence threshold for the multi-hop verdict and monitor the ratio of chain triggers to actual multi-hop questions in production traces. If you observe chains being executed for questions that a single retrieval round would have handled, raise the threshold or add a lightweight pre-check that tests whether a single retrieval call returns sufficient evidence before committing to the full chain.

This template is designed to be placed directly into your system prompt or a dedicated classification step before your retrieval pipeline executes. It forces the model to make a binary decision—single-hop or multi-hop—and provide a structured rationale. The strict JSON output contract with an enum-constrained dependency_verdict field makes the result trivially parsable by downstream orchestration logic, eliminating the need for fragile regex or intent-parsing on free-text responses.

textCopy
You are a query dependency classifier for a Retrieval-Augmented Generation (RAG) system. Your only job is to analyze a user's question and determine if it can be answered with a single round of retrieval or if it requires a chain of dependent queries where later queries need information from earlier results.

## Input
User Question: [USER_QUESTION]
Conversation History (last 3 turns): [CONVERSATION_HISTORY]
Available Data Sources: [AVAILABLE_DATA_SOURCES]

## Classification Rules
- Classify as "multi_hop" if answering the question requires finding one piece of information and then using that result to find another, distinct piece of information.
- Classify as "single_hop" if the question can be answered by retrieving documents matching the question's core entities and concepts in one retrieval call.
- Multi-hop indicators include: comparisons requiring separate lookups, questions with implicit intermediate entities, temporal sequences where one event depends on another, and questions using superlatives that require finding a set and then filtering.
- Do not classify a question as multi-hop simply because it is long or contains multiple clauses. Only classify as multi-hop if there is a genuine dependency chain.

## Output Contract
Respond with a single JSON object. No other text.
{
  "dependency_verdict": "single_hop" | "multi_hop",
  "confidence": 0.0-1.0,
  "rationale": "A concise explanation of why this verdict was chosen, citing specific phrases from the user question.",
  "suggested_hop_count": null | integer,
  "intermediate_entities": null | ["entity1", "entity2"]
}

## Constraints
- If confidence is below [CONFIDENCE_THRESHOLD], set dependency_verdict to "single_hop" and note the ambiguity in the rationale.
- Never invent entities not present or directly implied by the user question.
- If the conversation history resolves an ambiguity, reference it in the rationale.

To adapt this template, replace the square-bracket placeholders with your application's runtime values. [USER_QUESTION] is the raw user input. [CONVERSATION_HISTORY] should be a serialized summary of recent turns to resolve anaphora. [AVAILABLE_DATA_SOURCES] is a brief description of your indexes so the model can reason about whether a single retrieval target can satisfy the question. [CONFIDENCE_THRESHOLD] is a critical calibration parameter: start at 0.7 and adjust based on your tolerance for false-positive multi-hop classifications, which add latency and cost. After receiving the JSON response, validate the dependency_verdict enum strictly before routing to your single-hop or multi-hop retrieval pipeline. Log any parse failures and the raw model output for offline threshold tuning.

Integrating the dependency detection prompt into an application requires treating it as a pre-retrieval classifier, not a standalone utility. The prompt should be called immediately after user input is received and before any retrieval I/O is initiated. Its output—a binary verdict with a confidence score—acts as a traffic cop: a single-hop verdict routes the original query directly to your standard retrieval pipeline, while a multi-hop verdict triggers a more expensive, latency-sensitive chain-of-queries workflow. The implementation harness must enforce a strict timeout and cost ceiling at this stage because a false-positive multi-hop classification on a simple factual query wastes significant compute and degrades user experience. Wire the prompt into an asynchronous pre-processing step in your query intake service, and ensure the verdict is logged as a structured metadata field on every trace for later analysis.

The core of the harness is a validation and retry layer wrapped around the model call. The expected output is a JSON object with a dependency_verdict enum (single_hop or multi_hop), a confidence float between 0.0 and 1.0, and a rationale string. Implement a post-processing validator that rejects any response where the JSON schema is violated, the confidence score is outside bounds, or the rationale is empty. On validation failure, implement a single retry with a stricter prompt that includes the raw model output and the specific validation error. If the retry also fails, the system should default to the safest behavior for your application's risk profile—typically routing to the more thorough multi-hop path to avoid a false negative that could produce an incomplete answer. Crucially, you must calibrate a CONFIDENCE_THRESHOLD for your domain. Start with a conservative threshold like 0.8 for triggering multi-hop, and monitor the ratio of multi-hop verdicts that result in a single useful retrieval versus those that execute unnecessary chains. Adjust this threshold based on observed latency and cost data, not intuition.

Model choice and tool integration are critical harness decisions. This classification task benefits from fast, cost-effective models. Start with a capable small model like claude-3-haiku or gpt-4o-mini rather than a large frontier model, as the classification logic rarely requires deep reasoning. The prompt should be deployed without access to retrieval tools or external APIs; it relies solely on the user's question text. Log the raw user query, the model's verdict, confidence, and rationale, and the final routed path in a structured format for observability. Connect these logs to your evaluation framework to run regression tests weekly, checking for drift in the classifier's behavior. The next step after implementing this harness is to build a dashboard that tracks the multi-hop activation rate, average latency added by the classification step, and the downstream success rate of multi-hop chains versus single-hop retrievals. Avoid deploying this as a black box; its value is proven only when you can measure and tune the trade-off between answer completeness and system cost.