Conversation History Compression Prompt for Chat Agents

This prompt is designed for multi-turn chat and copilot systems where the raw conversation history has grown beyond practical token limits. The job-to-be-done is compressing a long transcript into a dense, structured summary that retains unresolved questions, explicit user corrections, key decisions, and active context while safely dropping turns that are resolved, redundant, or stale. The ideal user is an AI engineer or backend developer building a stateful chat product who needs a deterministic, testable compression step before the next inference call.

Use this prompt when your system maintains a full conversation log and you need to produce a compressed history block that fits inside a limited context window alongside system instructions, tool schemas, and retrieved evidence. It is appropriate for customer support copilots, long-running research assistants, and multi-turn agent workflows where losing track of an earlier user correction or an open question would degrade the user experience. The prompt expects a structured input containing the full conversation transcript and a set of compression directives—such as which categories of information to preserve and which to drop.

Do not use this prompt as a substitute for proper session state management in your application layer. If your product already tracks unresolved items, decisions, and corrections in a structured session object, you should assemble the compressed context programmatically rather than asking the model to reconstruct it from raw transcripts. This prompt is also not a replacement for a persistent memory system; it operates on a single session's history and does not carry user preferences or facts across sessions. For regulated domains where dropping any user statement could create compliance risk, pair this prompt with a risk-based retention check and human review gate before the compressed output is used downstream.

The prompt template below is designed to be inserted into your application's context assembly pipeline when the accumulated conversation history exceeds a token threshold you define. It instructs the model to act as a history compressor, producing a structured output that replaces the raw transcript. The template uses square-bracket placeholders for all dynamic inputs, making it straightforward to wire into a template engine or prompt builder. Before using this in production, you must define your own thresholds for when compression triggers, decide whether to run this synchronously before the next user turn or asynchronously in the background, and establish validation checks on the compressed output to ensure no critical information was dropped.

textCopy
You are a conversation history compressor for a multi-turn chat agent. Your task is to compress the provided conversation history into a structured summary that preserves all information necessary for the agent to continue the conversation accurately.

## INPUT

[CONVERSATION_HISTORY]

## COMPRESSION RULES

1. **Retain Unresolved Items**: Keep all user questions, requests, or issues that have not been fully resolved or answered. Include any partial progress toward resolution.
2. **Retain User Corrections and Feedback**: Preserve any instance where the user corrected the agent, expressed dissatisfaction, or provided explicit preferences. Note the correction and the agent's response.
3. **Retain Key Decisions and Facts**: Keep decisions made by the user or agent, confirmed facts, user-provided data (names, dates, numbers, constraints), and agreed-upon next steps.
4. **Retain Active Context**: Preserve the topic, goal, or task the user is currently working on. Include relevant entities, project names, or domain-specific terms.
5. **Drop Resolved Turns**: Remove turns where a question was fully answered and the user acknowledged or moved on without follow-up.
6. **Drop Greetings and Pleasantries**: Remove small talk, thank-yous, and conversational filler that carries no task information.
7. **Drop Redundant Repetition**: If the same information appears multiple times, keep only the most recent or most complete version.
8. **Preserve Temporal Order**: For retained items, maintain the chronological sequence of events, corrections, and decisions.

## OUTPUT FORMAT

Return a JSON object with the following schema:

{
  "summary": "A dense narrative summary of the conversation state, covering the current topic, what has been accomplished, and what remains open. Write in past tense. Maximum 3 sentences.",
  "unresolved_items": [
    {
      "item": "Description of the unresolved question, request, or issue",
      "status": "not_started | in_progress | blocked | awaiting_user",
      "context": "Any relevant context needed to resolve this item"
    }
  ],
  "key_decisions": [
    {
      "decision": "What was decided",
      "made_by": "user | agent | mutual",
      "timestamp_turn": "Approximate turn number or position reference"
    }
  ],
  "user_preferences_and_corrections": [
    {
      "correction_or_preference": "What the user corrected or stated as a preference",
      "agent_response": "How the agent acknowledged or adapted"
    }
  ],
  "active_context": {
    "topic": "Current topic or task",
    "entities": ["Key entities, names, or terms"],
    "constraints": ["Any constraints or requirements in play"]
  },
  "dropped_turns_summary": "Brief note on what types of turns were dropped and why (e.g., 'Dropped 4 turns of greetings and resolved Q&A')."
}

## CONSTRAINTS

- Do not invent information not present in the conversation history.
- If the history is already short and contains no resolved turns, return a minimal summary without forcing compression.
- If the conversation contains [SENSITIVE_DATA_POLICY], apply the redaction rules specified there before including any information in the output.
- Output ONLY the JSON object. No preamble, no commentary.

To adapt this template for your application, start by replacing [CONVERSATION_HISTORY] with your formatted transcript. The transcript format matters: include speaker labels (e.g., User:, Agent:) and turn boundaries so the model can distinguish who said what. The [SENSITIVE_DATA_POLICY] placeholder should be replaced with your organization's PII handling rules or removed if not applicable. You may also adjust the output schema fields to match your downstream agent's state representation. For example, if your agent uses a different state tracking structure, modify the JSON schema to produce objects your agent can consume directly without an additional transformation step. Test the compressed output by feeding it into a fresh conversation with your agent and verifying that the agent behaves as if it has full context for unresolved items while not re-litigating resolved ones.

Before deploying this prompt, build a small evaluation set of 5-10 conversation transcripts with known critical items that must survive compression. Run the prompt against each transcript and check that every unresolved question, user correction, and key decision appears in the output. Common failure modes include the model summarizing too aggressively and dropping a correction the user made three turns ago, or treating a partially resolved item as fully resolved because the agent said 'I'll look into that' but never reported back. If your application operates in a regulated domain, add a human review step for compressed histories before they replace the raw transcript in the agent's context window. Finally, log the compression ratio (input tokens vs. output tokens) and the dropped turns summary to monitor whether the compressor is being too aggressive or too conservative over time.

The compression prompt is not a standalone feature; it is a middleware step inside a stateful conversation loop. When the accumulated conversation history exceeds a configurable token threshold (for example, 80% of the model's context window), the application should trigger compression before the next turn. The uncompressed history is archived for audit and debugging, while the compressed summary replaces the full history in the active prompt context. This keeps the agent responsive and cost-effective without losing critical conversational state.

Wire the compression call as a synchronous pre-processing step before assembling the main agent prompt. Pass the full conversation array into the [CONVERSATION_HISTORY] placeholder, set [MAX_OUTPUT_TOKENS] to a fraction of your total context budget (typically 10-20%), and define [RETENTION_PRIORITIES] explicitly: unresolved questions, user corrections, key decisions, active constraints, and any domain-specific entities. The output should be a structured JSON object with compressed_summary (string), retained_turns (array of turn indices kept verbatim), and dropped_turns (array of indices removed with a one-line reason). Validate this JSON strictly before accepting the compressed result—if parsing fails, retry once with a stricter schema instruction, then fall back to truncating the oldest turns if the retry also fails.

Log every compression event with a unique compression_id, the pre-compression token count, post-compression token count, compression ratio, and the model version used. Store the full pre-compression history and the compressed output side by side in your observability pipeline. This traceability is essential for debugging quality regressions: if the agent starts missing context, you can replay the compression step and compare outputs. For high-stakes domains (healthcare, legal, finance), add a human-review gate on the first N compression events or on any compression that drops turns containing flagged keywords (e.g., 'allergy', 'cancel', 'dispute').

Model choice matters. Use a fast, cost-efficient model for compression (e.g., GPT-4o-mini, Claude Haiku, or a fine-tuned small model) rather than your primary reasoning model. The compression task is extractive and summarization-focused, not reasoning-intensive. If your application uses tool calls, ensure the compression prompt's [TOOLS_USED] placeholder captures the tool names and key outputs so the compressed history preserves the fact that a tool was called and what it returned, even if the full output is summarized. Finally, build a periodic eval that samples compressed histories and checks for fact preservation, unresolved question retention, and absence of hallucinated content—run this eval weekly or on every prompt version change to catch drift before users do.