History-Conditioned Query Reformulation Prompt for Conversational RAG

This prompt is designed for conversational AI engineers building stateful RAG systems where the user's current question cannot be understood in isolation. Its primary job is to resolve anaphora, ellipsis, and implicit references from the conversation history to produce a standalone, self-contained query. The ideal user is a developer integrating this prompt into a multi-turn retrieval pipeline, where a user might ask 'What about the pricing?' immediately after a discussion about a specific product. Without history-conditioned reformulation, the retrieval system would search for the generic term 'pricing' and fail to return the relevant document. This prompt is a pre-retrieval step that sits between the chat interface and the vector or keyword index.

You should use this prompt when your application supports back-and-forth dialogue and your knowledge base is large enough that contextless queries produce noisy or irrelevant results. It requires two inputs: the current user utterance and a structured representation of the conversation history, typically the last N turns. The output is a single, decontextualized query string. This prompt is not a substitute for session state management or long-term memory; it only resolves local conversational dependencies. Do not use it for single-shot Q&A, for injecting user profile data (use a profile-aware rewrite prompt instead), or when the conversation history contains sensitive information that should not be passed to the model. In high-compliance environments, ensure the history passed to the model is scrubbed of PII and access-controlled data before reformulation.

A critical constraint is topic boundary detection. The prompt must be instructed to ignore history that is no longer relevant. If a user discusses 'renewal terms' for ten turns and then asks 'How do I reset my password?', the reformulated query should be exactly 'How do I reset my password?' without dragging in the prior topic. The prompt template below includes explicit instructions for this. Before deploying, build a regression test suite with examples of pronoun resolution ('it', 'they'), comparative references ('the cheaper one'), and topic shifts. Evaluate the reformulated query against a golden set of expected standalone queries. If the reformulated query still contains pronouns or unresolved references, the downstream retrieval will fail silently, which is the most common production failure mode.

This prompt template is the core instruction set for a history-conditioned query reformulation step in a conversational RAG pipeline. It takes the user's latest message, a structured conversation history, and a user access scope, then produces a self-contained query that can be sent directly to a retrieval system. The prompt is designed to resolve anaphora (e.g., 'it', 'that feature'), ellipsis (e.g., 'what about the other one?'), and implicit topic carry-over without losing the user's original intent or introducing information not present in the history.

The template below uses square-bracket placeholders for all dynamic inputs. Replace [CONVERSATION_HISTORY] with a list of prior turns, each containing role and content. Replace [CURRENT_QUERY] with the user's latest message. Replace [USER_ACCESS_SCOPE] with a structured description of what the user is permitted to retrieve (e.g., document labels, project IDs, clearance levels). The [OUTPUT_SCHEMA] placeholder should be replaced with your expected JSON structure, such as {"standalone_query": "string", "resolved_references": ["string"], "access_constraints_applied": ["string"]}. The [FEW_SHOT_EXAMPLES] block is optional but strongly recommended for production; include 2-4 examples that demonstrate pronoun resolution, topic shifts, and access scope application.

textCopy
You are a query reformulation engine for a secure conversational retrieval system. Your job is to convert the user's latest query into a standalone, self-contained retrieval query that can be executed independently against a search index.

## Inputs
- Conversation History: [CONVERSATION_HISTORY]
- Current User Query: [CURRENT_QUERY]
- User Access Scope: [USER_ACCESS_SCOPE]

## Rules
1. Resolve all pronouns, demonstratives (this, that, these, those), and implicit references by substituting the specific entities or topics from the conversation history.
2. If the current query is a follow-up that omits the main subject, carry forward the subject from the most recent relevant turn.
3. Do NOT introduce new entities, constraints, or assumptions not present in the history or current query.
4. Apply the User Access Scope as a filter: if the resolved query would retrieve documents outside the user's permitted scope, add explicit scope-limiting terms or metadata constraints to the query.
5. If the user's query introduces a new topic unrelated to the history, treat it as a fresh query and do not force a connection to prior turns.
6. Preserve the user's original phrasing where possible; only modify what is necessary for standalone clarity and access compliance.

## Output Format
Return a JSON object matching this schema exactly:
[OUTPUT_SCHEMA]

## Examples
[FEW_SHOT_EXAMPLES]

Now reformulate the current query.

After copying this template, adapt the [USER_ACCESS_SCOPE] input to match your authorization model. If your system uses document-level ACLs, pass a list of allowed document IDs or label filters. If you use role-based scoping, describe the role's permitted knowledge domains. The [OUTPUT_SCHEMA] should include at minimum the standalone_query field; adding resolved_references and access_constraints_applied fields makes the reformulation auditable. Before deploying, run this prompt against a regression test set that includes pronoun chains, topic shifts, and queries that should be blocked by access scope. If the reformulated query still contains unresolved pronouns or leaks access-scoped terms, add more few-shot examples targeting those failure patterns rather than lengthening the rules.

This prompt is designed to be called synchronously as a pre-retrieval step in a conversational RAG pipeline. Before any search query hits your vector or keyword index, the raw user utterance and the conversation history must pass through this reformulation prompt. The output is a single, standalone, permission-aware query string that resolves anaphora, ellipsis, and implicit references. The application should treat this prompt as a deterministic transformation layer: it takes state (history, user profile) and an input (user message) and produces a cleaned output (reformulated query). Do not expose the raw reformulation to the end user unless you are building a debug or transparency panel.

To integrate this into your application, construct a request to your LLM provider with the prompt template as the system or user message. The [CONVERSATION_HISTORY] placeholder should be populated with a serialized list of prior turns, typically the last N messages formatted as User: ... Assistant: .... The [USER_QUERY] placeholder receives the current raw user input. The [USER_ACCESS_SCOPE] placeholder must be injected from your application's authentication layer, containing a structured summary of the user's roles, departments, and permission boundaries. After receiving the model's response, validate the output before retrieval: check that the reformulated query is a non-empty string, that it does not contain unresolved pronouns (e.g., 'it', 'they') unless they refer to entities explicitly named in the query itself, and that it does not introduce terms or access scopes beyond what is present in the user's profile. If validation fails, log the raw and reformulated queries, the history context, and the validation error, then either retry with a stricter temperature setting or fall back to using the raw user query with a warning flag.

For production resilience, implement a retry wrapper around this prompt call. Use a low temperature (0.0–0.2) to maximize deterministic reformulation. If the model returns a malformed response or the validator rejects the output, retry once with an explicit error instruction appended to the prompt, such as 'The previous output was invalid. Ensure the query is a single standalone sentence with no unresolved references.' Log every reformulation event—including the original query, the reformulated query, the conversation history window used, and the validator result—to a structured logging system. This log becomes essential for debugging retrieval failures and for building a regression test suite. For high-stakes domains where a bad reformulation could surface unauthorized documents, add a human review step for queries that trigger permission-boundary warnings or that the validator flags as ambiguous. Finally, treat this prompt as a versioned artifact in your codebase, with explicit release gates and a golden test set of conversation snippets that must pass before any prompt update is deployed.