Stale Turn Detection and Removal Prompt

This prompt is designed for a single, high-stakes job: identifying conversation turns that contain facts, assumptions, or retrieved evidence that have been explicitly or implicitly superseded by later information in the dialogue. The ideal user is a production AI engineer building a RAG-augmented copilot or a real-time assistant where a confident but wrong answer, based on outdated context, directly erodes user trust. You need this prompt when your system maintains a multi-turn history and ingests real-time data, such as a stock ticker, a live document, or a user's evolving preferences, where a fact stated five turns ago is now provably false.

This prompt is not a general-purpose history pruner. Do not use it for simple context window budgeting, where the goal is to drop low-signal turns to save tokens. For that, use a 'Conversation Pruning Decision Prompt' or 'Low-Value Turn Identification Prompt'. This prompt is specifically for staleness, which requires a comparative analysis: the model must contrast an earlier turn's claim against a later turn's correction or new evidence. It is also not a fact-checking prompt against a static knowledge base; it operates strictly within the provided conversation history. Use it when the cost of acting on stale context is high, such as executing a trade, sending a customer a wrong status update, or locking in a decision based on a reverted code change.

Before wiring this into your application, define a clear staleness taxonomy for your domain: 'superseded by user correction', 'contradicted by new retrieved evidence', 'assumption invalidated by new data', and 'explicitly retracted'. The prompt's output is a removal list, not a final action. Your application code should parse this list, log the staleness reasons for auditability, and then execute the removal. In high-risk domains like finance or healthcare, always route the removal list for human review before the pruned context is used to generate a user-facing response. A false positive here—removing a turn that only appeared to be stale—can be as damaging as a false negative. Start by evaluating this prompt on a golden dataset of conversations with known stale turns, measuring precision (did we flag only truly stale turns?) and recall (did we find all stale turns?) before deploying to production.

This prompt template is designed to be dropped into a RAG-augmented or real-time copilot pipeline where conversation history accumulates across turns and some prior statements, retrieved facts, or assumptions become invalid. The prompt instructs the model to scan the provided conversation history, compare each turn against the most recent evidence or user corrections, and flag turns that should be removed or re-verified before the next response is generated. The output is a structured removal list with reasons, not a rewritten conversation.

textCopy
You are a conversation state auditor. Your task is to examine the provided conversation history and identify turns that contain facts, assumptions, or retrieved evidence that have been superseded, contradicted, or invalidated by more recent information.

## INPUT

[CONVERSATION_HISTORY]

## INSTRUCTIONS

1. Process each turn in chronological order.
2. For each turn, determine whether any factual claim, retrieved evidence citation, or stated assumption has been:
   - Explicitly contradicted by a later user correction or assistant retraction.
   - Superseded by newer retrieved evidence or updated data.
   - Rendered irrelevant because the user changed topics and the turn contains only topic-specific context that no longer applies.
3. Do NOT flag turns solely because they are old. Age alone is not staleness.
4. Do NOT flag turns that contain only social niceties, acknowledgments, or structural dialogue (e.g., "Got it," "Thanks").
5. If a turn contains both stale and still-valid information, flag the entire turn but note which specific claims are stale.

## OUTPUT SCHEMA

Return a JSON object with the following structure:

{
  "stale_turns": [
    {
      "turn_index": <integer, zero-based index from [CONVERSATION_HISTORY]>,
      "staleness_type": "contradicted" | "superseded" | "topic_shifted",
      "stale_claims": ["<specific claim or fact that is stale>"],
      "contradicting_turn_index": <integer or null>,
      "reason": "<one-sentence explanation of why this turn is stale>",
      "recommended_action": "remove" | "reverify"
    }
  ],
  "retained_turns": [<array of turn indices that are safe to keep>],
  "summary": "<one-sentence summary of what changed and why>"
}

## CONSTRAINTS

- If no turns are stale, return an empty stale_turns array and include all turn indices in retained_turns.
- If you are uncertain whether a turn is stale, set recommended_action to "reverify" instead of "remove".
- Do not invent staleness. Only flag turns where the evidence of staleness is clear from the conversation itself.
- Preserve turns containing user corrections, even if the corrected content is flagged elsewhere.

## RISK LEVEL

[RISK_LEVEL]

## EXAMPLES

[EXAMPLES]

To adapt this template, replace [CONVERSATION_HISTORY] with your serialized conversation turns, including turn indices, speaker roles, and any attached evidence citations. The [RISK_LEVEL] placeholder should be set to high if the domain is regulated (healthcare, finance, legal) to bias the model toward reverify over remove. The [EXAMPLES] placeholder should contain one or two annotated few-shot examples showing correct staleness detection, including edge cases where a turn looks old but is still relevant. If your system already has a structured dialogue state, you can add a [CURRENT_STATE] field to give the model additional context about active goals and unresolved questions. After receiving the output, validate that every flagged turn_index exists in the input and that contradicting_turn_index references a later turn. For high-stakes deployments, route reverify recommendations to a human review queue rather than auto-removing those turns.

The stale turn detection prompt is not a standalone product feature; it is a middleware component that sits between your conversation state manager and your context assembly pipeline. In a typical RAG chat or copilot architecture, you will invoke this prompt after retrieving new evidence or receiving a user correction, but before assembling the final context for the next model call. The prompt expects a structured conversation history (each turn with speaker, content, and optional metadata like timestamps or source citations) plus a set of new facts, corrections, or retrieved evidence that may supersede earlier turns. Its output is a removal list with staleness reasons, which your application must parse and apply to the active session state before the next inference request.

To wire this into production, wrap the prompt in a function with a strict contract: accept a list of turns and a list of new evidence items, return a list of turn IDs to remove with reason codes. Validate the output schema before acting on it—reject any response that references turn IDs not present in the input, contains empty reason strings, or exceeds a configurable maximum removal threshold (removing more than 50% of turns in a single pass should trigger a human review or a conservative fallback that only removes the top-confidence stale turns). Log every removal decision with the turn ID, reason, and the new evidence that triggered it. This audit trail is essential for debugging confident-wrong-answer incidents and for tuning the staleness threshold over time.

Model choice matters here. This is a classification and reasoning task that benefits from models with strong instruction-following and structured output capabilities. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are all suitable. Avoid smaller or older models that may hallucinate turn IDs or produce inconsistent reason codes. Set temperature=0 or very low (0.1 maximum) to maximize output stability. If you are processing long conversations, consider chunking the history into overlapping windows and running detection on each window, then merging results with a deduplication step. For high-throughput systems, cache the prompt prefix (system instructions and output schema) and batch multiple sessions where latency allows.

The most dangerous failure mode is a false positive: removing a turn that contains critical context the user still relies on. To guard against this, implement a confidence threshold. The prompt should output a confidence score for each stale turn; your application should only auto-remove turns above a high threshold (e.g., 0.9) and flag borderline cases for human review or conservative retention. A second failure mode is stale detection lag—when the prompt fails to identify a superseded turn and the assistant continues to use outdated information. Mitigate this by running detection on every turn where new evidence is introduced, not just periodically. Monitor the rate of user corrections that reference previously removed context as a key metric; a rising rate indicates your staleness detection is too aggressive.

Do not use this prompt in isolation. Pair it with a context refresh step that re-retrieves or re-verifies facts when staleness is detected, rather than simply removing the stale turn and leaving a gap. Also ensure your session state serialization preserves the removal audit log so that if a session is paused and resumed, the pruned history is consistent. Finally, expose a manual override in your admin or support tooling: operators should be able to restore removed turns if a user reports that the assistant forgot something important. The prompt is a decision-support tool, not an irreversible action.