Session Summary Staleness Check Prompt

This prompt is designed for AI engineers managing long-running, multi-turn conversations where a compressed session summary is used as a proxy for full-turn history. Its primary job is to act as a gate: before the summary is injected into a downstream reasoning step, RAG retrieval query, or agent planning loop, it evaluates whether the summary still accurately reflects the current conversation state. The ideal user is a production AI builder who has already implemented session summarization and is now encountering failures caused by stale context—where the assistant confidently acts on outdated information because the summary hasn't been updated since a user correction or topic shift.

You should use this prompt when the cost of acting on stale context is high. Concrete examples include a customer support copilot that continues to reference a resolved issue as open, a coding agent that proposes edits against a file the user just said they deleted, or a research assistant that re-cites a source the user has explicitly discredited. In these scenarios, the prompt's structured output—a staleness score, a list of contradicted summary points, and a regeneration recommendation—provides a programmatic decision point. You can wire it so that a staleness score above a threshold automatically triggers summary regeneration before the next agent step, preventing cascading errors.

Do not use this prompt as a replacement for full-turn history when exact details are critical, such as in legal or clinical documentation workflows where paraphrasing is unacceptable. It is also not a general-purpose summarization prompt; it assumes a summary already exists and only checks its validity. Avoid running this check on every single turn in low-stakes, short-context chats where the overhead of the evaluation exceeds the cost of an occasional stale reference. The next section provides the copy-ready template you can adapt and integrate into your context assembly pipeline.

This prompt template is designed to be dropped into your staleness detection harness. It compares a previously generated session summary against the most recent turns of the full conversation history. The model's job is not to summarize the conversation again, but to act as an auditor: identify which summary points are contradicted, which are still supported, and whether the summary as a whole should be regenerated. Use this template as the core evaluation step in a pipeline that gates summary reuse.

textCopy
You are an auditor evaluating whether a session summary is still accurate given recent conversation turns.

[CONTEXT]
- Session Summary (previously generated): [SESSION_SUMMARY]
- Recent Conversation Turns (full text, most recent first): [RECENT_TURNS]
- Summary Generation Turn: The summary was generated after turn [SUMMARY_GENERATION_TURN_NUMBER].
- Current Turn: [CURRENT_TURN_NUMBER]

[CONSTRAINTS]
- Compare each factual claim in the session summary against the recent turns.
- A claim is contradicted if the user explicitly corrects it, provides new information that supersedes it, or the conversation has moved to a state that makes the claim no longer applicable.
- A claim is unsupported if it cannot be verified from the recent turns but is not explicitly contradicted.
- A claim is still valid if the recent turns confirm or are consistent with it.
- Do not mark a claim as contradicted just because the topic has shifted. Topic shift alone is not staleness.
- If the user has asked to start over, reset, or ignore prior context, treat all prior summary claims as contradicted.

[OUTPUT_SCHEMA]
Return a JSON object with the following structure:
{
  "staleness_score": number, // 0.0 (fully fresh) to 1.0 (fully stale)
  "summary_claims_evaluated": [
    {
      "claim": string, // the exact claim from the session summary
      "status": "valid" | "contradicted" | "unsupported",
      "evidence": string, // the specific turn text that supports the status, or "no supporting evidence found"
      "contradicting_turn": number | null // turn number that contradicts the claim, if applicable
    }
  ],
  "contradicted_claims": [string], // list of contradicted claim texts only
  "recommendation": "regenerate" | "retain" | "partial_update",
  "recommendation_reasoning": string, // brief explanation of the recommendation
  "regeneration_priority_claims": [string] // claims that must be updated if doing a partial update
}

[EXAMPLES]
Example 1: User corrects a prior statement.
Session Summary: "User is based in San Francisco and uses the Enterprise plan."
Recent Turn: "Actually, I moved to Austin last month."
Expected: The claim "User is based in San Francisco" is contradicted. The claim about the Enterprise plan remains valid if not addressed.

Example 2: User changes topic without invalidating prior context.
Session Summary: "User was troubleshooting a billing error on invoice #1234."
Recent Turn: "Can you also help me set up a new team member?"
Expected: The billing claim is still valid. Topic shift is not staleness. Recommendation: retain.

Example 3: User explicitly resets.
Session Summary: Contains multiple claims about a project.
Recent Turn: "Let's start over. I want to talk about something completely different."
Expected: All claims are contradicted. Recommendation: regenerate.

[RISK_LEVEL]
Medium. A false positive (recommending regeneration when the summary is fine) wastes compute and latency. A false negative (recommending retention when the summary is stale) causes the assistant to act on outdated information, eroding user trust. Always validate against ground truth from full-turn history.

To adapt this template, replace the placeholders with your actual data. [SESSION_SUMMARY] should be the compressed summary from a prior turn. [RECENT_TURNS] should be the raw conversation turns that occurred after the summary was generated. If your system does not track turn numbers, you can replace [SUMMARY_GENERATION_TURN_NUMBER] and [CURRENT_TURN_NUMBER] with relative timestamps or sequence markers. The output schema is designed to be machine-readable for downstream logic: use staleness_score to set thresholds, contradicted_claims to trigger targeted updates, and recommendation to decide whether to regenerate, retain, or patch the summary. For high-stakes domains such as healthcare or finance, route contradicted_claims to a human review queue before the assistant acts on the updated summary.

The Session Summary Staleness Check Prompt is designed to run as a pre-response gate in long-running conversational AI systems. Before the assistant generates a reply, the harness should pass the current session summary and the most recent N user and assistant turns to the staleness check prompt. The model returns a structured staleness assessment, including a score, a list of contradicted summary points, and a recommendation to regenerate or retain the summary. This output should be treated as a control signal—not shown to the user directly—that determines whether the downstream prompt receives the existing summary or triggers a fresh summarization pass.

Integration flow: (1) Load the current session summary from your state store. (2) Retrieve the last [TURN_WINDOW] turns from the full-turn history (ground truth). (3) Populate the prompt template with [SESSION_SUMMARY] and [RECENT_TURNS]. (4) Call the model with response_format set to the expected JSON schema containing staleness_score, contradicted_points, and recommendation. (5) Validate the output: confirm the score is a float between 0.0 and 1.0, each contradicted point references a specific summary claim, and the recommendation is one of regenerate, retain, or review. If validation fails, retry once with the validation error appended as a [CONSTRAINT] note. (6) If recommendation is regenerate or staleness_score exceeds your threshold (e.g., >0.6), trigger a summary regeneration prompt using the full-turn history before proceeding to response generation. Log the staleness score, recommendation, and contradicted points for observability.

Model choice and latency: This check adds a model round-trip before every response, so prefer a fast, smaller model (e.g., GPT-4o-mini, Claude Haiku, or a fine-tuned local model) for the staleness check itself. Reserve larger models for the actual response generation and summary regeneration steps. If latency is critical, run the staleness check in parallel with other pre-response operations, but ensure the response generation step waits for the staleness verdict. Ground-truth evaluation: Maintain a labeled dataset of session summaries paired with recent turns where you know whether the summary is stale. Run this prompt against that dataset regularly to calibrate your staleness threshold and detect drift in the model's sensitivity to contradictions. Measure precision (don't flag fresh summaries as stale) and recall (don't miss real staleness). False positives waste regeneration tokens; false negatives let stale context poison responses.

Failure modes to monitor: (1) The model may flag stylistic rephrasing as contradiction—validate that contradicted points represent factual disagreement, not wording differences. (2) Very long session summaries may exceed the context window when combined with recent turns—implement a truncation strategy that preserves the most recent summary sections and the full recent-turn window. (3) The staleness check itself can hallucinate contradictions that don't exist—always compare the output's contradicted_points against the actual turn history in your eval harness. (4) Rapid topic shifts can produce high staleness scores even when the summary is technically accurate for prior topics—consider adding a topic_shift_detected field to your output schema to distinguish staleness from intentional topic changes. Human review: For high-stakes domains (healthcare, legal, finance), route review recommendations to a human queue rather than auto-regenerating. Log every staleness decision with the full prompt, model response, and final action taken for auditability.