Conversation Drift vs Stale Context Classification Prompt

This prompt solves a specific decision problem that sits between conversation history analysis and context refresh execution. In multi-turn assistants and RAG-augmented copilots, the system must decide whether the user's latest turn represents an intentional topic change (drift) or whether the assistant is operating on outdated session assumptions (stale context). Getting this classification wrong has immediate production consequences: misclassifying drift as staleness triggers unnecessary context refreshes, re-verification loops, and re-retrieval that waste latency and compute budget. Misclassifying staleness as drift causes the assistant to confidently answer from invalid information—the exact failure mode that erodes user trust in retrieval-augmented systems.

Deploy this prompt in the decision layer before any context refresh, re-verification, or session state reset. The ideal user is an AI engineer or product developer building a multi-turn assistant where context freshness directly impacts answer quality. Required inputs include the full conversation history, the current session summary or state object, and the user's latest turn. The prompt works best when the assistant already maintains structured dialogue state—slot values, pending actions, retrieved facts, and active assumptions—because the classification needs to compare the user's new intent against what the system currently believes to be true. Without structured state, the prompt can still operate on raw conversation history, but precision drops because it cannot distinguish between a user correcting a specific stale fact versus pivoting to a new topic that happens to overlap in vocabulary.

Do not use this prompt when the cost of a false negative (missing stale context) is catastrophic and the cost of a refresh is trivial. For example, a medical copilot displaying patient lab results should refresh on any ambiguity rather than risk showing outdated values. Similarly, skip this classification step when the conversation is single-turn, when the assistant has no persistent state, or when the user explicitly requests a refresh—explicit user commands bypass the classification layer entirely. This prompt also underperforms when the conversation history is too short to establish a baseline (fewer than three substantive turns) because drift and staleness are indistinguishable without a pattern of prior context. For those cases, default to a lightweight staleness heuristic based on time elapsed since last retrieval rather than a full classification pass.

This template provides the core classification logic for distinguishing between two common but easily confused conversation states: intentional topic drift (the user is deliberately changing the subject) and stale context (the assistant is operating on outdated session facts, retrieved evidence, or prior-turn assumptions that no longer hold). The prompt forces the model to produce a structured label with explicit reasoning, which is essential for downstream routing—drift should trigger context reset, while staleness should trigger re-verification or refresh. Without this distinction, assistants either cling to irrelevant context when users move on, or discard valid context when a single fact becomes outdated.

textCopy
You are a conversation state classifier for a production chat assistant. Your job is to analyze the current user turn against the assistant's active session context and determine whether the user is intentionally changing topics (drift) or the assistant is relying on outdated information (stale context).

[CONTEXT]
- Current user turn: [USER_TURN]
- Last assistant response: [LAST_ASSISTANT_RESPONSE]
- Active session summary: [SESSION_SUMMARY]
- Recently retrieved evidence (if any): [RETRIEVED_EVIDENCE]
- Conversation phase: [CONVERSATION_PHASE]

[OUTPUT_SCHEMA]
Return a JSON object with these fields:
- "classification": "drift" | "stale_context" | "both" | "neither"
- "confidence": 0.0 to 1.0
- "reasoning": A concise explanation referencing specific evidence from the user turn and session context.
- "drift_signals": Array of strings describing what indicates intentional topic change (empty if none).
- "stale_signals": Array of strings describing what context appears outdated or contradicted (empty if none).
- "recommended_action": "reset_context" | "refresh_context" | "ask_clarification" | "continue"
- "context_items_to_refresh": Array of specific session facts, retrieved passages, or assumptions that need re-verification (empty if none).

[CONSTRAINTS]
1. A user correction ("No, that's wrong...") is a staleness signal, not drift, unless the user explicitly changes topics afterward.
2. A user asking a new but related question is drift only if the prior context is no longer relevant to the new question.
3. If the user's turn contradicts a fact in the session summary or retrieved evidence, flag that specific fact as stale.
4. If the user's turn introduces a completely new domain, entity, or task unrelated to the last 3 turns, classify as drift.
5. "Both" applies when the user changes topics AND the assistant's prior context contains outdated information that would pollute the new topic.
6. "Neither" applies when the user continues the same topic and all context remains valid.
7. Do not classify as stale simply because retrieved evidence is old—only if the user's turn or new information contradicts it.
8. Confidence below 0.7 should trigger "ask_clarification" as the recommended action.

[EXAMPLES]
Example 1 - Drift:
User turn: "Actually, let's switch gears. What's the status of the Q3 marketing campaign?"
Session summary: Assistant was helping user debug a Python deployment error.
Classification: drift
Reasoning: User explicitly signals topic change with "switch gears" and introduces a new domain (marketing) unrelated to prior debugging context.

Example 2 - Stale Context:
User turn: "I already told you we moved the database to PostgreSQL last week."
Session summary: Assistant referenced MySQL configuration in prior response.
Classification: stale_context
Reasoning: User corrects a factual assumption (database type) that the assistant carried forward from earlier turns. The topic hasn't changed—the context is outdated.

Example 3 - Both:
User turn: "Forget the deployment issue. Also, we migrated to Kubernetes last month, so your Docker Compose suggestion won't work."
Session summary: Assistant was troubleshooting a deployment failure using Docker Compose assumptions.
Classification: both
Reasoning: User abandons the deployment topic (drift signal: "forget the deployment issue") and corrects an infrastructure assumption (stale signal: migration to Kubernetes invalidates Docker Compose context).

[RISK_LEVEL]
High. Misclassifying drift as staleness causes unnecessary re-verification and user frustration. Misclassifying staleness as drift causes context loss and forces the user to repeat information. In regulated domains, stale context that goes undetected can produce factually incorrect outputs with compliance implications.

Adaptation guidance: Replace the square-bracket placeholders with live data from your conversation state manager. [SESSION_SUMMARY] should be a compressed representation of prior turns, not raw history—use your session summarization prompt to generate this. [RETRIEVED_EVIDENCE] should include source identifiers and retrieval timestamps so the model can reason about recency. [CONVERSATION_PHASE] helps the model calibrate its sensitivity (e.g., in an onboarding phase, apparent drift may actually be normal exploration). The [EXAMPLES] section is critical for few-shot performance—add 2-3 examples from your own domain where drift and staleness are commonly confused. For high-stakes deployments, route confidence < 0.7 outputs to a human reviewer or a secondary classification model before taking action. Test this prompt against a labeled dataset where drift and staleness are intentionally interleaved; expect precision above 0.85 and recall above 0.80 before removing the clarification fallback.

This classification prompt is designed to sit between the conversation history manager and the response generation step in a chat pipeline. It should be invoked when the assistant is about to use prior context to answer a new user turn. The prompt receives the last N turns of conversation history and the assistant's current context assumptions, then outputs a classification label (drift, stale_context, or neither) with reasoning. The classification determines whether the pipeline should carry forward existing context, refresh it, or treat the turn as a new topic. Wire this as a synchronous pre-processing step before the main generation call, not as an asynchronous post-hoc analysis.

Validation and Retry Logic: Parse the model output against a strict schema expecting classification (enum: drift, stale_context, neither), confidence (float 0-1), and reasoning (string). If the output fails schema validation, retry once with the same prompt plus the validation error message appended as a [VALIDATION_ERROR] block. If the second attempt also fails, default to neither with a confidence of 0.0 and log the failure for review. For high-stakes domains (healthcare, legal, finance), route stale_context classifications with confidence below 0.85 to a human review queue before allowing context refresh or user-facing responses. Model Choice: Use a fast, instruction-tuned model (e.g., GPT-4o-mini, Claude 3.5 Haiku) for this classification step to keep latency low; the classification itself should not add more than 500ms overhead. Logging: Log every classification decision with the conversation turn ID, the classification label, confidence score, and the full reasoning string. This trace is essential for debugging false positives (drift classified as staleness) and false negatives (stale context missed) during eval cycles.

Integration with Context Refresh: When the classifier outputs stale_context, trigger your context refresh workflow before generating the assistant response. This may involve re-querying your RAG pipeline, asking the user a clarification question, or regenerating the session summary. When the classifier outputs drift, reset or down-weight prior context assumptions and treat the turn as a new topic boundary. When neither, proceed with existing context. Avoid coupling the classification prompt directly to the refresh action—instead, use the classification as a signal that downstream logic interprets. This separation lets you tune refresh behavior (e.g., cost-aware refresh budgets, user clarification preferences) without modifying the classification prompt. Testing in Production: Deploy this prompt behind a feature flag and run it in shadow mode for at least 1000 turns, comparing its classifications against manual labels before letting it control context behavior. Measure precision and recall on both stale_context and drift labels separately, as the cost of misclassifying staleness as drift (user gets outdated information) is typically higher than the reverse (unnecessary context refresh).