Context Pruning with Importance Scoring Prompt Template

This prompt is designed for AI platform architects and application developers who need to recover from a context window overflow or token limit exhaustion during a long-running task. The core job-to-be-done is to intelligently reduce the size of the context payload before a retry, rather than blindly truncating or starting over. The ideal user is someone building an automated retry harness who needs the model to act as its own editor: scoring each piece of context by its relevance to the current task and removing low-scoring content. This is most effective when the original task failed with a clear 'context length exceeded' error and you have a structured list of context elements (e.g., retrieved documents, conversation turns, code files) that can be individually scored and pruned.

Do not use this prompt when the failure is caused by something other than context length, such as a malformed tool call, a policy refusal, or a factual hallucination. It is also the wrong tool if you cannot break your context into discrete, scorable items—if your context is a single monolithic block of text, a summarization or compression prompt is a better fit. This prompt requires a well-defined [TASK_DESCRIPTION] and a list of [CONTEXT_ELEMENTS] with unique identifiers. Without these, the importance scoring will be unreliable. For high-stakes domains like healthcare or legal review, the pruning output must be logged and available for human audit to ensure no critical evidence was removed before the retry.

After using this prompt, you must validate the pruned context against your original task constraints. Check that the total token count of the remaining elements fits within your model's context window with room for the system prompt and expected output. Implement a circuit breaker: if the pruned context still exceeds the limit after scoring, either escalate to a human or fall back to a more aggressive summarization strategy. The next step is to feed the pruned context into your original task prompt and evaluate whether the output quality and factual grounding are preserved compared to a full-context baseline.

The prompt below is designed to be inserted into a retry harness when a model response fails due to context window overflow, token limit exhaustion, or output truncation. Instead of blindly truncating the oldest content, this template instructs the model to score each context element by its importance to the current task, remove the lowest-scoring items, and re-attempt the original request with the pruned context. This approach preserves critical instructions, key facts, and unresolved dependencies while making room for a complete response.

textCopy
You are a context pruning assistant. Your job is to reduce the size of the provided context so that it fits within a [TOKEN_BUDGET] token limit while preserving the information most critical to completing the original task.

## ORIGINAL TASK
[TASK_DESCRIPTION]

## FULL CONTEXT (too large to process)
[FULL_CONTEXT]

## INSTRUCTIONS
1. Review the ORIGINAL TASK and identify what information is essential to complete it.
2. Score each distinct element in the FULL CONTEXT on a scale of 1-5 based on its relevance to the task:
   - 5: Critical — the task cannot be completed without this information.
   - 4: High relevance — directly supports task completion.
   - 3: Moderate relevance — provides useful context but is not essential.
   - 2: Low relevance — tangentially related.
   - 1: Irrelevant — can be safely removed.
3. Remove all elements scored 1-2. If the remaining context still exceeds [TOKEN_BUDGET] tokens, remove elements scored 3, starting with the least relevant.
4. Never remove elements scored 5. Only remove elements scored 4 if absolutely necessary and note what was removed.
5. Output the pruned context in the same format as the original, preserving section structure where possible.

## OUTPUT FORMAT
```json
{
  "pruned_context": "<pruned context text>",
  "removed_elements": [
    {"element_summary": "<brief description>", "score": <1-5>, "reason": "<why removed>"}
  ],
  "estimated_token_count": <number>,
  "critical_elements_preserved": true
}

CONSTRAINTS

• Preserve all [CRITICAL_ENTITIES] such as names, dates, IDs, and code variables.
• Preserve all unresolved questions, pending decisions, and action items.
• If [OUTPUT_SCHEMA] is provided, ensure pruned context still contains all information needed to produce valid output.
• Do not fabricate or summarize content that was not in the original context. Only remove.

To adapt this template, replace the square-bracket placeholders with your application's values. [TOKEN_BUDGET] should be set to the remaining tokens available after accounting for the system prompt, this pruning instruction, and the expected output length. [TASK_DESCRIPTION] should be the exact original task the model was attempting. [FULL_CONTEXT] is the complete context that overflowed. [CRITICAL_ENTITIES] is an optional list of entity types that must survive pruning (e.g., customer IDs, contract numbers, function signatures). [OUTPUT_SCHEMA] is an optional JSON Schema or format description that the final output must satisfy. Before deploying, test this prompt with known overflow scenarios and verify that the pruned context still enables correct task completion. If the task is high-risk—such as legal, financial, or clinical work—require human review of the pruning decisions before re-attempting the original request.

The context pruning prompt is not a standalone artifact—it is a recovery step inside a larger orchestration loop. The typical integration point is a context window overflow handler: when the primary task fails due to token limits, the harness captures the original prompt, the full context payload, and the error metadata, then invokes the pruning prompt to produce a reduced context. The pruned output is fed back into the original task prompt for a retry attempt. This means the harness must preserve the original task instruction, output schema, and any tool definitions across the pruning-and-retry cycle so that the retry evaluates the same task with a leaner context.

Wire the pruning prompt as a function with a clear contract: accept [ORIGINAL_TASK], [FULL_CONTEXT], [TOKEN_BUDGET], and [IMPORTANCE_CRITERIA] as inputs; return a structured JSON object containing pruned_context, removed_items with removal reasons, and an importance_scores map. Validate the output shape before using it. If the pruned context still exceeds the budget, apply a progressive reduction loop—re-invoke the pruning prompt with a stricter budget or higher importance threshold, but cap retries at three attempts to avoid infinite loops. Log every pruning decision for auditability: which context elements were removed, their scores, and the final token count. For high-stakes domains such as healthcare or legal review, route pruned outputs to a human approval queue before the retry executes.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities—GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro are suitable defaults. Avoid smaller models that may collapse the scoring into uniform values or hallucinate removal justifications. Set response_format to JSON mode or use structured output APIs to enforce the schema. On the harness side, implement a token counter (tiktoken or equivalent) to verify the pruned context fits the budget before retrying the original task. If the pruning prompt itself fails—malformed JSON, missing required fields, or a pruned context that still overflows—fall back to a simpler truncation strategy (keep the first N tokens or the most recent turns) and escalate to an operator. The goal is graceful degradation, not silent data loss.