Sequential Retrieval Query Planning Prompt

This prompt is a tactical planning tool for an agentic retrieval loop. Its sole job is to analyze a complex user question alongside the currently retrieved context, identify the specific evidence that is still missing, and generate a single, targeted search query to fill that gap. The ideal user is a RAG system operator or AI engineer who has already decomposed a complex question or started a reasoning chain and has received an initial batch of incomplete results. You should use this prompt when your system needs to decide what to retrieve next, not when you need to rewrite a user's initial query for better recall. It assumes you are inside a planning step, after an initial retrieval and before the next retrieval call is executed.

To use this prompt effectively, you must provide it with the original user question and the full text of the currently retrieved context. The prompt's value is in its precision: it forces the model to articulate the evidence gap before generating a query, making the agent's next action auditable. The output is not a final answer, a sub-question for a human, or a general topic suggestion. It is a concrete, self-contained search query designed to be passed directly to your vector, keyword, or hybrid search engine. A well-formed output will often include specific entities, technical terms, or temporal constraints that were absent from the initial retrieval.

Do not use this prompt as a standalone question-answering system or as a replacement for an initial query rewriter. It is not designed to handle the first retrieval step, where the goal is broad recall. It is also not a substitute for a full task decomposition prompt; it assumes a decomposition or reasoning plan is already in progress and only needs the next tactical move. If your system lacks a mechanism to track what has already been retrieved and what remains unknown, this prompt will produce queries that are redundant or misaligned with the overall plan. Use it as one component in a stateful agent loop, where the output query is executed, the new results are added to the context, and the cycle repeats until an evidence sufficiency check passes.

This prompt is designed to be the planning step in an agentic retrieval loop. Given a complex user question and the evidence gathered so far, its sole job is to identify the most critical missing piece of information and formulate a precise, self-contained search query to retrieve it. The template uses square-bracket placeholders for all dynamic inputs, making it straightforward to integrate into a Python or TypeScript harness where these variables are populated at runtime from the application state.

textCopy
You are a query planning specialist for a research assistant. Your task is to analyze a complex user question and the context that has already been retrieved. Identify the single most critical piece of missing information required to answer the question, and formulate a specific, self-contained search query to find it.

USER QUESTION:
[USER_QUESTION]

CURRENT RETRIEVED CONTEXT:
[RETRIEVED_CONTEXT]

PLANNING CONSTRAINTS:
- Do not re-ask the original question.
- Do not ask for information already present in the current context.
- The query must be a standalone search string, not a conversational turn.
- If the context is sufficient to answer the question, respond with "SUFFICIENT".
- If the question is unanswerable even with more information, respond with "UNANSWERABLE: [reason]".

OUTPUT FORMAT:
Return a JSON object with the following keys:
{
  "status": "NEXT_QUERY" | "SUFFICIENT" | "UNANSWERABLE",
  "reasoning": "A brief explanation of the gap identified and why this query is the best next step.",
  "next_query": "The specific search query string, or null if status is not NEXT_QUERY."
}

To adapt this template, replace the placeholders with your application's state. [USER_QUESTION] is the original, complex question from the user. [RETRIEVED_CONTEXT] is the concatenated text of all documents or chunks retrieved in previous steps. In your harness, you should parse the JSON response and use the status field to control the agent loop: if NEXT_QUERY, execute the search and feed the new results back into the context for the next iteration; if SUFFICIENT, proceed to answer generation; if UNANSWERABLE, surface the reason to the user or a human reviewer. For high-stakes domains, log the reasoning field alongside every step to create an auditable trail of the system's research path.

This prompt is designed as the reasoning step inside an agentic retrieval loop. It should not be used in isolation or as a one-shot call. The application harness must manage the loop state, track the evidence chain, and decide when to stop querying and synthesize a final answer. The prompt's job is narrow: given the current state of knowledge and a specific evidence gap, produce the next retrieval query. The harness owns the broader workflow—initial decomposition, gap detection, query execution, result ingestion, and termination logic.

Wire the prompt into a loop controller that maintains a structured evidence log. Before each call, populate [ORIGINAL_QUESTION] with the user's initial query, [CURRENT_KNOWLEDGE_SUMMARY] with a concise synthesis of all evidence retrieved so far, and [IDENTIFIED_GAP] with a specific, actionable description of what's missing. The model returns a JSON object with a search_query field and a rationale field. Validate the output before executing the query: reject empty queries, queries that are verbatim copies of previous attempts, or queries that don't address the stated gap. Implement a maximum iteration limit (typically 3-5 hops) and a semantic deduplication check against prior queries to prevent loops. Log every query, the retrieved results, and the gap assessment for debugging and audit trails.

For model choice, use a capable instruction-following model with strong reasoning performance (e.g., Claude 3.5 Sonnet, GPT-4o). Set temperature to 0.1-0.2 to balance creativity in query formulation with consistency. If the application is high-stakes—legal, clinical, financial—add a human review gate after the loop completes but before the final answer is shown to users. The review interface should display the original question, the full chain of queries and retrieved evidence, and the planned final synthesis. Do not treat this prompt as a black box; its output quality depends directly on the accuracy of the gap description fed into it. Invest engineering effort in the gap detection step that precedes this prompt in the loop.