Conversation History Compression Prompt for Long Sessions

This prompt is designed for AI engineers and platform teams operating long-running chat assistants, copilots, or multi-turn agent workflows where session history routinely exceeds the model's context window. The core job-to-be-done is converting a verbose, multi-turn transcript into a dense, information-preserving summary that fits within a strict token budget. You should reach for this prompt when your session state surpasses 80% of the available context window, when latency begins to degrade due to prompt length, or when you need a portable session payload for handoff to another agent, storage in a session database, or pause-and-resume functionality. The ideal user is a developer integrating this prompt into an automated context management pipeline, not an end-user manually summarizing a chat.

Do not use this prompt for short, single-turn interactions or sessions that comfortably fit within the context window with all instructions and tool outputs intact. It is also the wrong tool when you need a human-readable narrative for a support ticket or a meeting notes summary; those workflows require different output schemas and salience criteria. This prompt is specifically tuned for machine-to-machine state compression where the primary consumer is another model or a structured session store. Using it for general-purpose summarization will produce overly dense, salience-scored payloads that omit the narrative flow a human reader expects.

Before deploying this prompt, you must have a clear token budget defined and a mechanism for triggering compression (e.g., a context window utilization threshold). The prompt includes salience scoring instructions that rank each turn by decision relevance, factual content, and future utility. This ranking enables deterministic pruning when the summary itself approaches the budget limit. The output is a structured compression payload, not free-text prose, so your downstream consumer must expect that schema. If your application cannot consume structured summaries, adapt the output format before integrating this prompt into your pipeline.

The most common production failure occurs when critical facts are dropped during compression without detection. This prompt includes a faithfulness verification harness that compares the compressed summary against the original transcript to flag hallucinations, incorrect attributions, and missing binding decisions. In high-stakes domains such as healthcare, legal, or financial services, you must route flagged summaries for human review before the compressed state replaces the full history. Never treat compression as lossless; always preserve the original transcript in cold storage for audit and recovery purposes.

The following prompt template is designed to be dropped directly into your application's context management layer. It instructs the model to act as a conversation archivist, transforming a long, multi-turn dialogue into a structured summary. The core mechanism is a salience-scoring instruction that forces the model to prioritize decision-critical facts, pending actions, and unresolved questions over social niceties or redundant information. The template uses square-bracket placeholders for all dynamic inputs, allowing you to inject the raw transcript, your specific token budget, and any domain-specific constraints before each call.

textCopy
You are a precise conversation archivist. Your task is to compress the provided [CONVERSATION_HISTORY] into a dense, structured summary that fits within a strict token budget of [MAX_OUTPUT_TOKENS].

Follow these rules:
1.  **Salience Scoring:** Mentally score each turn for its long-term utility. Prioritize: explicit user decisions, confirmed facts, unresolved questions, action items with owners, and critical corrections. Discard: greetings, small talk, redundant confirmations, and information explicitly superseded by a later correction.
2.  **Structured Output:** Generate the summary in the following JSON schema:
    {
      "session_objective": "A single sentence describing the user's primary goal.",
      "key_decisions": ["A list of final, confirmed decisions made."],
      "critical_facts": ["A list of verified facts provided by the user or confirmed from tools."],
      "pending_actions": [{"action": "...", "owner": "user or assistant", "status": "open"}],
      "unresolved_questions": ["Questions the user asked that were not answered."],
      "corrections_log": [{"superseded_claim": "...", "correction": "...", "turn_id": "..."}]
    }
3.  **Faithfulness Constraint:** Do not invent any facts, decisions, or questions not explicitly present in [CONVERSATION_HISTORY]. If a field has no valid entries, return an empty list `[]`.
4.  **Budget Compliance:** If the JSON output exceeds [MAX_OUTPUT_TOKENS], iteratively prune the least salient items from the `critical_facts` and `key_decisions` lists until the budget is met. Do not truncate mid-JSON.

[CONVERSATION_HISTORY]:
"""
[RAW_TRANSCRIPT]
"""

[DOMAIN_TERMINOLOGY]:
"""
[OPTIONAL_GLOSSARY]
"""

To adapt this template, start by replacing [RAW_TRANSCRIPT] with your actual conversation log, ideally with turn IDs and speaker roles (user, assistant, tool) clearly labeled. The [MAX_OUTPUT_TOKENS] placeholder should be set dynamically based on your context window budget—a common starting point is 15-25% of the total context limit. The [OPTIONAL_GLOSSARY] field is a powerful lever for domain-specific compression; providing a list of acronyms and entity definitions here prevents the summary from wasting tokens on verbose explanations. For high-stakes applications, you must pair this prompt with a faithfulness verification harness that programmatically compares the generated summary against the original transcript to detect dropped critical facts or hallucinated entries before the summary replaces the full history.

This prompt is designed to sit inside a context window management middleware that triggers compression when the accumulated conversation history approaches a configurable token threshold (e.g., 80% of the model's context limit). The harness should intercept the raw turn list, inject the prompt template with the conversation history, token budget, and salience rubric, and then replace the verbose history with the compressed summary in subsequent requests. Do not call this prompt on every turn—trigger it only when the token counter signals that headroom is running out. The compression output becomes the new system or context preamble for the next request, with the most recent 2-3 turns retained in full to preserve immediate coherence.

Validation and retry logic is critical because a corrupted or hallucinated summary will poison every subsequent turn. After the model returns a compressed summary, run a faithfulness verification prompt (see sibling topic: Summary Faithfulness Verification Prompt) that compares the summary against the original transcript and flags any hallucinated facts or dropped critical information. If the verification fails—defined as any hallucinated claim or a completeness score below your threshold—retry the compression prompt once with the verification errors appended as [CONSTRAINTS]. If the retry also fails, log the full transcript and alert a human operator; do not silently proceed with a degraded summary. For high-stakes domains (healthcare, legal, finance), require human review of the compressed summary before it replaces the live context.

Model choice and cost considerations matter here. Use a fast, cheaper model (e.g., GPT-4o-mini, Claude Haiku) for the compression step itself, reserving your primary high-capability model for the actual user-facing turns. The compression prompt's output should be a structured JSON object with compressed_summary, salience_scores, and dropped_turn_ids fields. Parse this JSON in your application layer, validate the schema, and store the dropped_turn_ids in your session metadata for auditability. Log every compression event with: session ID, token count before/after, compression ratio, verification pass/fail, and latency. This telemetry is essential for tuning your trigger thresholds and detecting silent degradation over long sessions.

Wiring into a stateful session store is the final integration step. The compressed summary should be persisted alongside the session record so that if the user disconnects and resumes later, the restart flow can load the summary rather than replaying the full history. Use the compressed_summary field as the [CONTEXT] input for the Session Restart Summary Prompt. Avoid the common failure mode of double-compression: if a session has already been compressed once, do not feed the compressed output back into the compression prompt as if it were raw history. Track a compression_count in session metadata and cap it at one or two levels before forcing a full context reset with a user-facing message.