Topic Shift Detection Prompt for RAG Assistants

Multi-turn RAG assistants degrade when stale context pollutes retrieval and answer generation. This prompt detects when a user changes subjects so your application can trigger a context reset, clear conversation history, or re-retrieve evidence. Use it as a guard before your main RAG pipeline to decide whether the current turn continues the prior topic or starts a new one. It classifies topic shifts as abrupt, gradual, or none, and provides a confidence score and boundary marker. This is not a follow-up question detector. It specifically identifies semantic topic boundaries, not anaphora or ellipsis resolution.

Deploy this prompt when your assistant handles open-domain conversations where users may pivot from troubleshooting to billing to technical specs without warning. It is most valuable in customer support copilots, internal knowledge base assistants, and any multi-turn system where context window pollution from irrelevant history degrades answer quality. The prompt works best when you have at least two prior turns to compare against the current message. Single-turn or stateless RAG systems do not need this detection layer. If your assistant operates in a narrow domain where topic changes are rare or predictable, a simpler keyword-based router may be more cost-effective than an LLM-based shift detector.

Avoid using this prompt when your primary need is resolving pronouns or follow-up references. Those require anaphora resolution, not topic boundary detection. Do not deploy this as the sole guard for context management—pair it with a context budget monitor and a staleness timer. For high-stakes domains where missed topic shifts could produce misleading answers, add a human review step when the confidence score falls below your threshold. Start by wiring this prompt before your retrieval step, log every shift classification, and measure whether context resets actually improve downstream answer accuracy before making it a permanent part of your pipeline.

This prompt template is designed to be placed between conversation turns in a RAG assistant. Its job is to analyze the user's latest message against the recent conversation history and determine whether the user has shifted to a new, unrelated topic. The output is a structured decision that your application harness can use to clear stale context, flush prior evidence, and trigger a fresh retrieval pipeline. Use this when your assistant suffers from 'context clinging'—answering new questions with old, irrelevant documents.

textCopy
SYSTEM: You are a topic continuity monitor for a multi-turn RAG assistant. Your only job is to detect topic shifts.

INPUT:
- [CONVERSATION_HISTORY]: The last [N] turns of the dialogue, each with a user message and assistant response.
- [CURRENT_USER_MESSAGE]: The user's latest message.
- [TOPIC_BOUNDARY_SENSITIVITY]: A value of 'strict', 'moderate', or 'loose'.

TASK:
Determine if [CURRENT_USER_MESSAGE] represents a topic shift from the conversation in [CONVERSATION_HISTORY]. A topic shift means the user has introduced a new subject that is not a follow-up, clarification, or natural continuation of the prior turns. Consider:
- Abrupt shifts: The user asks about a completely different domain (e.g., from 'deployment errors' to 'lunch plans').
- Gradual shifts: The user moves to a tangentially related but distinct subject (e.g., from 'Python performance' to 'hardware costs').
- False shifts: The user uses a pronoun or reference that ties back to the prior topic, even if the wording is new.

OUTPUT_SCHEMA:
{
  "topic_shift_detected": boolean,
  "confidence": 0.0-1.0,
  "reasoning": "string explaining the decision in one sentence",
  "recommended_action": "reset_context" | "re_retrieve" | "continue"
}

CONSTRAINTS:
- If [TOPIC_BOUNDARY_SENSITIVITY] is 'strict', classify any non-explicit follow-up as a shift.
- If 'loose', only classify a shift when the subject domain changes entirely.
- If 'moderate', use your best judgment for tangential shifts.
- Do not answer the user's question. Only output the JSON.

To adapt this template, replace the placeholders with your application's actual values. [CONVERSATION_HISTORY] should be a serialized list of recent turns, typically the last 3-5 exchanges. [N] is a tunable parameter; a smaller window is more sensitive to recent shifts, while a larger window provides more context for detecting gradual transitions. [TOPIC_BOUNDARY_SENSITIVITY] should be set based on your application's tolerance for unnecessary re-retrieval. A 'strict' setting is appropriate for focused technical assistants where context contamination is costly. A 'loose' setting works for open-domain chatbots. After pasting, wire the JSON output into your application's state machine: a topic_shift_detected: true with recommended_action: reset_context should clear the conversation history and the retrieved document cache before the next turn.

The topic shift detection prompt is a classification gate that sits between the user's latest message and the rest of your RAG pipeline. Its job is to decide whether the conversation has changed subjects enough to require a context reset and re-retrieval. This prompt should be called on every user turn before you assemble the final context for answer generation. The output is a structured decision—typically a JSON object with a topic_shift boolean, a confidence score, and an optional new_topic_label—that your application code reads to branch between two paths: continue with existing context or flush and re-retrieve.

Wire this prompt into your application as a pre-processing step in the turn loop. After receiving user input, call the LLM with the topic shift prompt, passing the last N turns of conversation history and the current user message as [CONVERSATION_HISTORY] and [CURRENT_MESSAGE]. Parse the response with a strict JSON validator. If the model returns malformed JSON, retry once with a repair prompt that includes the raw output and the expected schema. If topic_shift is true and confidence exceeds your threshold (start with 0.7), trigger a context reset: clear the retrieved evidence cache, re-run retrieval with the new message as a standalone query, and reset the conversation history to only the system prompt and the current turn. If confidence is below threshold, log the ambiguous case for review and default to keeping context—but flag the turn for potential re-retrieval if the next answer quality eval scores low.

Model choice matters here. This is a classification task that benefits from fast, cheap models. Use a lightweight model like GPT-4o-mini, Claude Haiku, or Gemini Flash for this gate. Reserve your larger reasoning model for the actual answer generation step. Add observability: log every topic shift decision with the confidence score, the turns compared, and whether a reset was triggered. This data becomes invaluable for tuning the confidence threshold and identifying patterns—like users who bounce between related topics without a true shift—that cause false positives. For high-stakes domains where a missed topic shift could produce dangerously irrelevant answers, add a human review queue for low-confidence decisions or implement a secondary check: after answer generation, run a quick relevance eval to catch cases where the topic shift detector failed and trigger a retroactive reset.