Conversation History Pruning Prompt

This prompt is for production engineers and AI operators who need to keep long-running agent or copilot sessions within a strict context window budget. The job-to-be-done is automated, turn-level salience scoring: for each turn in a conversation history, the prompt assigns a relevance score relative to the user's current goal and provides a retention rationale. The output is a structured pruning plan that tells your application which turns are safe to drop and which must be preserved because they carry critical state, unresolved questions, or active constraints. This is not a generic summarization prompt—it is a budget-allocation tool for systems where dropping the wrong turn breaks task completion.

Use this prompt when your application manages sessions that routinely exceed the model's context limit, when you need deterministic, auditable pruning decisions rather than opaque summarization, and when you must preserve state-carrying utterances (slot values, pending actions, policy constraints) while discarding low-signal turns like greetings, acknowledgments, or redundant clarifications. The prompt is designed for chat assistants, copilots, and multi-turn agents where context pollution from stale turns degrades response quality. It is also appropriate when you need a pruning trace for debugging or compliance, since each turn receives an explicit score and rationale rather than a black-box summary.

Do not use this prompt for short sessions that comfortably fit within the context window, for real-time voice interactions where latency constraints preclude per-turn scoring, or when your application already maintains structured dialogue state externally and can reconstruct context without replaying history. This prompt is also not a substitute for session summarization when the goal is to produce a human-readable digest; use a dedicated summarization prompt for that. In high-risk domains—healthcare, legal, finance—the pruning decisions should be logged and reviewed, and critical state-carrying turns should be flagged for human verification before permanent removal. The prompt is a component in a larger context budget management system, not a standalone solution.

This prompt template is designed to be inserted into a larger prompt assembly pipeline. It receives a full or partial conversation history and a description of the current user goal, then returns a scored, filtered, and pruned history that retains only the turns most relevant to the active task. The template uses square-bracket placeholders for all dynamic inputs, making it safe to template with standard string-replacement tools before injection into a model request.

textCopy
You are a conversation history pruner for a production AI assistant. Your job is to reduce the token cost of a conversation history while preserving all information necessary to complete the current user goal correctly.

You will receive:
- A conversation history as a list of turns, each with a speaker label and message text.
- A description of the current user goal.
- A target maximum turn count [MAX_TURNS].
- A list of critical state fields that must be preserved: [CRITICAL_STATE_FIELDS].

For each turn in the history, assign:
- A relevance score from 0.0 to 1.0, where 1.0 means the turn is essential to the current goal.
- A retention rationale in one short sentence.
- A list of state fields carried or modified by this turn, drawn only from [CRITICAL_STATE_FIELDS].

Then, produce a pruned history that:
- Contains at most [MAX_TURNS] turns.
- Retains all turns that carry or modify any field in [CRITICAL_STATE_FIELDS].
- Retains the highest-scoring remaining turns until the budget is filled.
- Preserves the original turn order.
- Replaces pruned turns with a single summary turn: {"speaker": "system", "message": "[Pruned N turns: summary of dropped content in one sentence]"}.

Return your output as a JSON object with this exact schema:
{
  "scored_turns": [
    {
      "turn_index": 0,
      "speaker": "user",
      "relevance_score": 0.95,
      "retention_rationale": "Contains the primary task instruction.",
      "state_fields_affected": ["intent", "target_entity"]
    }
  ],
  "pruned_history": [
    {"speaker": "user", "message": "..."},
    {"speaker": "assistant", "message": "..."}
  ],
  "pruning_summary": {
    "original_turn_count": 24,
    "pruned_turn_count": 8,
    "turns_dropped": 16,
    "critical_state_preserved": true,
    "estimated_token_savings_percent": 60
  }
}

Conversation history:
[HISTORY]

Current user goal:
[USER_GOAL]

To adapt this template, replace the square-bracket placeholders with values from your application context. [HISTORY] should be a serialized list of turn objects with speaker and message keys. [USER_GOAL] should be a concise natural-language description of what the user is currently trying to accomplish, extracted from the most recent turns or from an upstream intent classifier. [MAX_TURNS] is an integer budget derived from your context window allocation. [CRITICAL_STATE_FIELDS] is a list of field names your application cannot afford to lose—such as ["intent", "account_id", "pending_action"]—drawn from your dialogue state schema. After receiving the model output, validate the JSON structure, confirm that all critical-state-carrying turns are present in the pruned history, and check that the turn count does not exceed the budget. If validation fails, retry with a stricter constraint or fall back to keeping the most recent [MAX_TURNS] turns. For high-stakes workflows where state loss could cause incorrect actions, route pruning decisions above a confidence threshold to a human reviewer or log the full scored output for audit.

The Conversation History Pruning Prompt is not a standalone utility; it is a stateful component in a context budget management pipeline. In production, this prompt sits between your conversation store and the model request assembler. Before each inference call, the harness retrieves the full conversation history, passes it through the pruning prompt along with the current user goal and context window budget, and receives a pruned history that preserves state-carrying turns while dropping low-salience utterances. The pruned output is then fed into the main assistant prompt. This means the pruning prompt's latency and token cost must be accounted for in your end-to-end budget—if pruning costs 500 tokens to save 2,000 tokens, the net savings must justify the additional inference step.

The implementation harness requires several guard layers. First, schema validation: the pruning prompt must return a structured JSON object containing the pruned turn list, a list of dropped turn IDs, and a pruning rationale per dropped turn. Validate this schema before accepting the output—if the model returns malformed JSON or references turn IDs that don't exist in the input, reject the output and retry with a stricter schema instruction. Second, state integrity verification: after pruning, run a lightweight check that critical state elements (confirmed slots, pending actions, unresolved entities) are still present in the retained turns. If a slot value was established in turn 4 and turn 4 is dropped, the harness must either reject the pruning decision or inject a summary of that slot into the retained context. Third, budget enforcement: count the tokens in the pruned output and verify it falls under the target budget. If the model fails to prune aggressively enough, fall back to a simpler truncation strategy (e.g., keep system prompt + last N turns) rather than silently exceeding the context window.

For model choice, use a fast, cost-efficient model for the pruning step—this is a classification and selection task, not a generation task. A smaller model like GPT-4o-mini, Claude Haiku, or a fine-tuned open-weight model works well here. Reserve your larger reasoning model for the main assistant task that consumes the pruned history. Implement retry logic with a maximum of two attempts: if schema validation fails, retry with the error message injected into the prompt. If state integrity fails, do not retry—log the failure and fall back to a safe truncation strategy. Log every pruning decision with the dropped turn IDs, retention rationale, and state integrity check result for debugging and eval. When deploying in high-stakes domains (healthcare, legal, finance), add a human review sampling step: periodically pull pruning decisions for manual review to catch systematic over-pruning of critical context before it affects users.

Avoid wiring this prompt directly into every inference call without caching. If the conversation history hasn't changed since the last pruning run, reuse the previous pruned result. Only re-run pruning when new user or assistant turns are appended. Also avoid pruning too early in a session—if the total history is already under the context budget, skip pruning entirely to eliminate unnecessary latency and cost. The pruning threshold should be configurable: only activate when history exceeds, for example, 70% of the available context window after accounting for system instructions and tool outputs.