Turn Importance Scoring Prompt for Pruning

This prompt is for production AI engineers and chat application developers who need a fine-grained, auditable method for managing context window budgets in long-running conversations. The core job-to-be-done is to assign a retention priority and a justification to each turn in a dialogue history, distinguishing between turns that carry critical state (e.g., a confirmed user ID or a resolved decision), turns that provide clarifying context, and turns that are safe to drop without breaking the assistant's ability to complete the current task. The ideal user is someone who has already implemented a basic context management strategy—such as a sliding window or a simple summarization step—and now needs a more precise, token-efficient pruning mechanism that can be evaluated and tuned against human judgment.

You should use this prompt when your application's context budget is under pressure from long session histories, when naive truncation strategies are causing the assistant to forget critical facts or repeat questions, or when you need to produce a verifiable pruning decision that can be logged for debugging and audit. The prompt is designed to be run as a pre-processing step before the main model call, scoring each turn so that a downstream application harness can drop the lowest-scoring turns until the payload fits within a target token limit. It is particularly effective in task-oriented assistants, customer support copilots, and multi-step workflow agents where some turns contain durable state (slot values, confirmed decisions) and others are purely transitional.

Do not use this prompt as a real-time, per-token classifier in a streaming architecture; it is designed for batch scoring of complete turns before assembly. It is also not a substitute for a proper session state object or a structured slot-tracking system—if you already have a reliable external state store, you may not need turn-level importance scoring at all. Avoid using this prompt on conversations shorter than five turns, where the overhead of scoring exceeds the benefit of pruning. Finally, this prompt is not a content safety or policy filter; it scores turns for relevance and state-carrying importance, not for toxicity or compliance. For high-stakes domains such as healthcare or legal workflows, always log the pruning decisions and retain the full history outside the model context for audit purposes.

This prompt template assigns an importance score and retention rationale to every turn in a multi-turn conversation. It is designed for production AI engineers who need to decide which turns to keep when the context window budget is exceeded. The prompt distinguishes between turns that carry critical state (e.g., slot values, decisions, pending actions), turns that provide clarifying context, and turns that are safe to drop without degrading task completion. Use this template as the core of a pruning pipeline that runs before context assembly.

textCopy
You are a conversation analyst. Your task is to score each turn in the following conversation history for its importance to the current task.

[CONVERSATION_HISTORY]

[OUTPUT_SCHEMA]

For each turn, output a JSON object with the following fields:
- turn_id: string (the identifier from the conversation)
- importance_score: integer (1-5, where 1 = safe to drop, 5 = must retain)
- retention_rationale: string (one sentence explaining why this score was assigned)
- carries_state: boolean (true if this turn contains slot values, decisions, pending actions, or confirmed facts)
- is_clarifying: boolean (true if this turn resolves ambiguity or provides disambiguating context)
- safe_to_prune: boolean (true if dropping this turn would not harm task completion)

Scoring guidelines:
- Score 5: Turns that establish the user's goal, contain critical slot values, record decisions, or list pending actions.
- Score 4: Turns that provide essential constraints, corrections, or clarifications that change the assistant's understanding.
- Score 3: Turns that add helpful but non-essential context or examples.
- Score 2: Turns that are acknowledgments, small talk, or redundant restatements.
- Score 1: Turns that are greetings, farewells, or completely irrelevant to the current task.

[CONSTRAINTS]
- Do not hallucinate turn content. Only reference information present in the provided history.
- If a turn is ambiguous, score conservatively (higher) and note the ambiguity in the rationale.
- The sum of importance scores is not constrained. Score each turn independently.
- Output only valid JSON. No markdown, no commentary outside the JSON.

To adapt this template for your application, replace [CONVERSATION_HISTORY] with your serialized turn data, including turn IDs, speaker roles, and timestamps. Replace [OUTPUT_SCHEMA] with any additional fields your pruning logic requires, such as slot_references or dependency_turn_ids. The [CONSTRAINTS] block should be updated to reflect your specific risk tolerance: in high-stakes domains like healthcare or finance, add a rule that any turn with a confidence score below a threshold must be flagged for human review rather than pruned. Always validate the output JSON against your schema before using the scores to prune, and log any turns where the model's safe_to_prune flag contradicts the importance_score for manual inspection.

This prompt is designed to be called by an application-level context budget manager, not directly by the end user. The typical integration point is inside a pre-request hook that fires before the main model call. When the accumulated conversation history exceeds a token threshold (e.g., 80% of the model's context window), the harness invokes this scoring prompt with the full turn history and a target retention ratio. The returned scores and rationales then drive a deterministic pruning step that drops the lowest-scoring turns until the budget is met. The harness must treat the prompt output as advisory: the final decision to drop a turn belongs to the application, not the model.

Validation and retry logic is critical because a malformed or hallucinated score can drop a turn that carries essential state. The harness should validate that the output is valid JSON, that every turn ID in the input appears exactly once in the output, that scores are within the specified range, and that the retention rationales are non-empty strings. If validation fails, retry once with the same input and a stronger constraint instruction appended. If the retry also fails, fall back to a simpler recency-based pruning strategy and log the failure for review. For high-stakes workflows (e.g., healthcare, legal, or financial assistants), route validation failures to a human review queue rather than silently pruning.

Model choice and latency matter here because this prompt runs on every pruning event. Use a fast, cost-efficient model (e.g., a small hosted model or a quantized open-weight model) for the scoring step, reserving larger models for the main generation task. The scoring prompt is a classification-style task with structured output, which smaller models handle well. Cache the prompt prefix (the system instruction and output schema) to reduce per-call latency and cost. Log every pruning event with the turn IDs dropped, their scores, and the retention rationales so you can audit pruning decisions later and tune the scoring threshold.

State integrity verification is the final safety net. After pruning, run a lightweight check that the remaining turns still contain all required slot values, pending actions, and unresolved entities. If the pruned history is missing critical state, restore the highest-scoring dropped turn that contains the missing information and re-run the verification. This prevents the pruning step from silently breaking downstream task completion. Wire this verification into your eval pipeline: measure task completion rate with and without pruning to confirm that importance scoring preserves the turns that actually matter for user goals.