Gemini Conversation History Packing for Long Sessions Template

This prompt is for AI engineers and product developers building extended, stateful chat sessions with Google's Gemini models. The core job-to-be-done is programmatic context management: you need to pack a long, multi-turn conversation history into a single model request without exceeding Gemini's token limits or losing critical decision-making context. The ideal user is someone who has already moved past a simple stateless prompt and is now hitting token-limit errors, observing degraded answer quality in long sessions, or paying too much for redundant context. You should use this prompt when your application maintains a session object with a growing list of user and model turns, and you need a reliable, automatable way to decide what to keep, what to summarize, and what to drop before the next inference call.

This playbook is not a general-purpose chat prompt. Do not use it if you are building a single-turn Q&A system, a stateless API wrapper, or a RAG workflow where the primary context comes from a vector store rather than a conversation log. It is also the wrong tool if your session history is short enough to fit entirely within the model's context window with room to spare—in that case, simple truncation by recency is cheaper and less complex. The prompt template assumes you have already implemented a session store and can programmatically inject the packed history into the contents array of the Gemini API request. It does not replace the need for a system instruction; instead, it focuses exclusively on the history-packing step that happens before the final prompt assembly.

Before implementing this prompt, confirm that your application genuinely needs long-session memory. If users rarely exceed 20 turns, a sliding window of the last N messages may be sufficient. Use this playbook when you observe that stale turns are polluting the model's attention, key facts from early in the conversation are being forgotten, or your token costs are dominated by redundant history. The next section provides the copy-ready template. Wire it into your application's pre-inference pipeline, and pair it with the token budget compliance checks and context retention quality tests described later in this playbook.

This prompt template is designed to be inserted into a Gemini chat session when the conversation history approaches the model's context limit or a predefined token budget threshold. Its job is to instruct the model to compress the preceding conversation turns into a dense summary that preserves critical decisions, unresolved questions, and user preferences, while discarding stale or redundant exchanges. The output is a replacement system or user message that can be prepended to a fresh session, effectively resetting the context window without losing the session's state.

textCopy
You are a conversation history compressor for a long-running Gemini chat session. Your task is to read the provided conversation history and produce a dense, structured summary that fits within a [MAX_OUTPUT_TOKENS] token budget. The summary must retain the following:

- [CRITICAL_DECISIONS]: All explicit decisions, agreements, or conclusions reached by the user and the assistant.
- [UNRESOLVED_QUESTIONS]: Any open questions, pending tasks, or items the user explicitly asked to revisit.
- [USER_PREFERENCES]: Stated user preferences, constraints, or stylistic choices that should govern future responses.
- [CONTEXTUAL_FACTS]: Key facts, names, dates, or code snippets that are essential for continuing the work.

You must discard:
- Greetings, pleasantries, and conversational filler.
- Turns that were fully resolved and have no bearing on future work.
- Redundant information that was later corrected or superseded.

Output the summary in the following [OUTPUT_SCHEMA] format as a JSON object:
{
  "session_summary": "A dense narrative paragraph covering all retained items.",
  "key_decisions": ["decision 1", "decision 2"],
  "open_items": ["open question 1"],
  "user_preferences": ["preference 1"],
  "token_count_estimate": <integer>
}

If the conversation history is empty or contains no information worth retaining, return a JSON object with empty arrays and a null session_summary.

[CONVERSATION_HISTORY]

To adapt this template, replace the [MAX_OUTPUT_TOKENS] placeholder with a concrete integer that leaves enough room in the context window for the next turn's system instructions and user input. The [CRITICAL_DECISIONS], [UNRESOLVED_QUESTIONS], [USER_PREFERENCES], and [CONTEXTUAL_FACTS] placeholders should be customized to match the specific domain of your application—for example, a coding agent might prioritize file paths and error traces, while a research assistant might prioritize source citations. The [CONVERSATION_HISTORY] placeholder should be replaced with the raw, serialized conversation turns. Before deploying, run this prompt against a golden dataset of conversations with known key decisions and verify that the compressed output includes all critical items and omits all stale turns. A common failure mode is the model summarizing too aggressively and dropping a user preference that was stated early in the session; mitigate this by adding a specific instruction to scan the entire history for preferences before summarizing.

This prompt is not a standalone chat instruction; it is a context compression function that should be called programmatically when a session's token count approaches a defined threshold. The application layer is responsible for tracking the token count of the raw conversation history, deciding when to invoke the packing prompt, and replacing the full history with the packed summary for subsequent turns. The prompt template expects a [FULL_CONVERSATION_HISTORY] and a [TOKEN_BUDGET] as inputs, and it must return a structured JSON object containing packed_history, decisions_log, and stale_turns_removed fields. Do not call this prompt on every turn; it is a state-mutating operation that should be triggered by a token threshold (e.g., 80% of the model's context limit) or a turn-count heuristic.

Validation and retry logic is mandatory. The application must parse the model's JSON response and validate it against a strict schema before replacing the live conversation state. If the packed_history field is missing, malformed, or exceeds the [TOKEN_BUDGET], the application should not use the output. Instead, implement a retry loop with a backoff strategy: re-invoke the prompt with an explicit error message appended to the [FULL_CONVERSATION_HISTORY], such as 'The previous summary was invalid. You must produce a valid JSON object with a packed_history field that stays under the token budget.' After two failed retries, the application should fall back to a simpler truncation strategy (e.g., dropping the oldest turns) and log the failure for debugging. For high-stakes sessions where decision retention is critical, route packing failures to a human review queue before the session continues.

Model choice and tool use are key implementation decisions. This prompt is designed for Gemini models with native context caching, so place the system-level packing instructions in a cacheable system message to reduce latency and cost on repeated calls. The packing prompt itself should be a single-turn developer message with the conversation history injected as a user message. Do not use function calling or tool-use for this operation; the model must return the packed state directly as structured JSON. For observability, log the token counts before and after packing, the number of turns removed, and the schema validation result. This data is essential for tuning the packing threshold and detecting drift in model behavior over time.