Session Summarization Prompt for Long Conversations

This prompt is designed for RAG system operators and AI engineers who manage long-running chat or copilot sessions where the full conversation history exceeds the model's context window. The core job-to-be-done is to produce a lossy, structured compression of a dialogue that preserves resolved questions, outstanding items, key facts, and user intent. The ideal user is someone integrating this into a context assembly pipeline—they need a summary that can be injected into future turns as a stable, high-signal prefix without breaking citation chains or introducing hallucinated facts. You should use this prompt when a session has more than 10-15 turns, when token budgets are tight, or when you need to checkpoint state before a context window reset.

Do not use this prompt for short conversations where the full history fits comfortably in context; the compression overhead and risk of information loss outweigh the benefit. It is also unsuitable for real-time streaming applications where latency is critical, as summarization adds a non-trivial processing step. The prompt assumes you have access to the full conversation transcript and any previously generated summaries. It does not perform retrieval or answer questions—it only compresses existing dialogue. If your system requires evidence re-retrieval after summarization, pair this with a stale context expiration check. The output must be treated as a lossy artifact; always preserve the original transcript for audit trails and debugging.

Before implementing this prompt, define your compression ratio target and the specific information categories your application cannot lose—such as user PII, unresolved action items, or source citations. Build eval checks that compare answer quality with and without the summary injected into context. Test for information preservation by asking the model questions about facts from the compressed portion of the conversation and measuring recall. The next step is to wire this prompt into your context assembly harness with validation that the output schema is respected and that no hallucinated turns appear in the summary.

This prompt template produces a structured session summary from a long conversation transcript and its associated RAG evidence. It is designed to be injected into future context windows so the assistant can maintain continuity without carrying the full conversation history. The template uses square-bracket placeholders that you replace with actual values before each call. Every placeholder represents a variable your application layer must supply—do not leave them unresolved in production requests.

codeCopy
You are a session summarization system for a RAG assistant. Your task is to compress a long conversation into a structured summary that preserves essential information for future context windows.

## INPUT

### Conversation Transcript
[CONVERSATION_TRANSCRIPT]

### Previously Retrieved Evidence (if any)
[RETRIEVED_EVIDENCE]

### Prior Session Summary (if any)
[PRIOR_SESSION_SUMMARY]

## OUTPUT SCHEMA

Return a JSON object with the following structure:

{
  "session_id": "[SESSION_ID]",
  "summary_timestamp": "[CURRENT_TIMESTAMP]",
  "resolved_questions": [
    {
      "question": "string",
      "answer_summary": "string",
      "source_citations": ["citation_id_1", "citation_id_2"],
      "resolution_confidence": "high|medium|low"
    }
  ],
  "outstanding_items": [
    {
      "item": "string describing unresolved question or task",
      "reason_unresolved": "insufficient_evidence|user_deferred|clarification_needed|out_of_scope",
      "partial_progress": "string or null"
    }
  ],
  "key_facts_established": [
    {
      "fact": "string",
      "source": "citation_id or 'user_stated'",
      "relevance": "high|medium|low"
    }
  ],
  "user_intent_summary": "string describing the user's overall goal and any shifts in intent",
  "conversation_turns_summarized": <integer>,
  "compression_notes": {
    "original_turns": <integer>,
    "summarized_turns": <integer>,
    "information_loss_risk": "low|medium|high",
    "dropped_topics": ["string"] or []
  }
}

## CONSTRAINTS

[CONSTRAINTS]

## INSTRUCTIONS

1. Read the full conversation transcript and any prior session summary.
2. Identify which questions have been definitively answered with cited evidence.
3. Identify which questions remain unresolved and why.
4. Extract key facts that were established during the conversation, noting their source.
5. Summarize the user's overall intent and any topic shifts.
6. Count the conversation turns and note any information that could not be preserved.
7. If a prior session summary exists, merge it with new information rather than duplicating.
8. Do not fabricate answers or citations. If evidence is missing, mark the item as unresolved.
9. Keep the summary concise but complete. Prefer precision over brevity.
10. Return ONLY the JSON object. No markdown fences, no commentary.

## EXAMPLES

[EXAMPLES]

Adapting the template: Replace [CONVERSATION_TRANSCRIPT] with the full dialogue history, including user messages, assistant responses, and any tool outputs. [RETRIEVED_EVIDENCE] should contain the passages that were retrieved during the session, with their citation IDs preserved. [PRIOR_SESSION_SUMMARY] is optional—pass null or an empty object if this is the first summarization. The [CONSTRAINTS] placeholder lets you inject domain-specific rules such as maximum summary length, required fields, or compliance requirements. [EXAMPLES] should contain one or two few-shot examples showing correct output format for your domain.

Validation and risk: After the model returns the JSON, validate that resolved_questions entries include source citations and that outstanding_items have a non-empty reason_unresolved field. For high-stakes domains such as healthcare or legal, route summaries with information_loss_risk: "high" to human review before they enter the next context window. Track compression ratio (summarized_turns / original_turns) over time—if it drops below 0.3, your summary may be too lossy and you should increase the token budget or adjust the constraints. Always log the summary alongside the original transcript for auditability.

The session summarization prompt is not a standalone feature; it is a context management utility that runs as a background step inside a multi-turn RAG application. Its job is to compress a long conversation history into a structured summary that can be injected into future context windows, preserving resolved questions, outstanding items, and key facts while discarding redundant turns. The implementation harness must treat this prompt as a deterministic pipeline stage with measurable success criteria: information preservation rate, compression ratio, and downstream answer quality after summary injection.

Wire the prompt into your application as a scheduled or threshold-triggered step. When the conversation token count exceeds a configured limit (e.g., 80% of the model's context window), extract the full dialogue history and pass it to the summarization prompt along with a structured output schema. Validate the returned JSON for required fields such as resolved_items, outstanding_questions, key_facts, and compression_metadata. If validation fails, retry once with a stricter schema reminder in the prompt. Log every summary generation event with the input token count, output token count, compression ratio, and validation status for observability. For high-stakes domains, route summaries flagged with low confidence or missing fields to a human review queue before they enter the active context window.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities for the summarization step, even if your primary RAG model is different. The summary must be injected into subsequent turns as a system-level context block, clearly demarcated from live retrieved evidence and the current user query. Never allow the summary to silently overwrite or contradict freshly retrieved passages. Implement a stale-summary check: if the summary is more than N turns old or the user has explicitly corrected information contained in it, trigger a re-summarization. Avoid the temptation to summarize too aggressively; a summary that drops critical unresolved questions will silently degrade answer quality across future turns, and this failure mode is hard to detect without explicit eval checks on downstream answer accuracy.