Context Window Overflow Retry with Reduced Context Prompt Template

This playbook is for application developers and AI engineers who need an automatic, deterministic recovery strategy when a model request fails due to a context window overflow error. The core job-to-be-done is to programmatically re-attempt the original task by systematically reducing the input context size, rather than simply failing the request or asking the user to manually shorten their input. The ideal user is someone building a production AI harness where long-context workflows—such as document analysis, extended chat sessions, or large-codebase reasoning—are common and where manual intervention is not scalable.

You should use this prompt when your application receives a specific 'context_length_exceeded' or equivalent token limit error from the model API. It is designed for scenarios where the original prompt assembly logic can be re-executed with a reduced context budget, and where you have defined a clear priority ordering for context sections (e.g., system instructions are more critical than the middle of a chat history). This approach is most effective when the context can be broken into discrete, rankable blocks. Do not use this prompt if the overflow is caused by a single, indivisible piece of content that cannot be logically split (like one massive PDF page), or if the task requires perfect recall of all original context. In those cases, consider a 'Summarize-Then-Continue' or 'Sliding Window Summary' strategy instead.

Before implementing, you must define a priority schema for your context sections. A typical priority order is: [SYSTEM_INSTRUCTIONS], [RECENT_USER_MESSAGES], [TOOL_OUTPUTS], [OLDER_MESSAGES], [FEW_SHOT_EXAMPLES]. The retry harness will use this schema to progressively drop the lowest-priority sections until the request fits within the model's token limit. A common failure mode is dropping critical instructions early, which causes the retry to succeed technically but produce a useless output. Always validate that the highest-priority sections are preserved in the first retry attempt, and implement a hard stop after a maximum of 3-5 progressive reductions to prevent infinite loops. For high-stakes tasks, log the full original and reduced prompts for offline debugging and eval.

This template is designed to be invoked by your application harness immediately after receiving a context-length error from the model API. Its job is not to perform the original task, but to reconstruct a smaller, higher-priority version of the request that fits within the model's limits. The prompt instructs the model to act as a context reduction engine: it receives the original task, the full context that caused the overflow, and a priority-ordered list of context sections. It must then drop the lowest-priority sections first, summarize where possible, and output a clean, self-contained prompt that can be sent as a fresh request.

textCopy
You are a context reduction engine. Your only job is to produce a compact, self-contained prompt that fits within a [MAX_OUTPUT_TOKENS] token budget.

## ORIGINAL TASK
[TASK_DESCRIPTION]

## ORIGINAL CONTEXT (CAUSED OVERFLOW)
[FULL_CONTEXT_THAT_OVERFLOWED]

## PRIORITY-ORDERED CONTEXT SECTIONS (highest first)
[PRIORITY_ORDERED_SECTION_LIST]

## REDUCTION RULES
1. Preserve all sections marked as CRITICAL without modification.
2. For HIGH priority sections, keep the full text unless summarization is required to meet the budget.
3. For MEDIUM priority sections, aggressively summarize to 1-2 sentences each, preserving only named entities, numbers, dates, and explicit constraints.
4. Drop all LOW priority sections entirely.
5. If the budget is still exceeded after dropping LOW sections, progressively summarize MEDIUM sections, then HIGH sections, but never drop or summarize CRITICAL sections.
6. The output must be a single, valid prompt that includes the original task instruction and the reduced context. Do not output explanations, apologies, or meta-commentary.

## OUTPUT FORMAT
Output only the reconstructed prompt, ready for immediate use as the next model request. Begin with the task instruction, followed by the reduced context.

To adapt this template, replace the placeholders with runtime values from your application. [TASK_DESCRIPTION] should contain the exact instruction the original request was trying to execute. [FULL_CONTEXT_THAT_OVERFLOWED] is the complete prompt that triggered the context-length error. [PRIORITY_ORDERED_SECTION_LIST] is the critical customization point: your application must tag each logical section of the original context with a priority level (CRITICAL, HIGH, MEDIUM, LOW) based on your domain knowledge. For example, in a RAG workflow, the user's question and safety policies might be CRITICAL, retrieved passages might be HIGH, conversation history might be MEDIUM, and optional formatting examples might be LOW. [MAX_OUTPUT_TOKENS] should be set to the model's context limit minus a safety margin for the task instruction itself. After receiving the reduced prompt, your harness should validate that it is syntactically complete and does not itself exceed the token limit before sending it as the retry request. For high-stakes domains, log both the original and reduced prompts for auditability, and consider a human-review step if the reduction drops HIGH-priority sections.

The Context Window Overflow Retry with Reduced Context prompt is designed to be called automatically when your application receives a context-length error from the model provider. The harness should catch the error, classify it as a recoverable overflow (as opposed to a permanent content policy or auth failure), and invoke the retry prompt with a reduced version of the original context. This is not a prompt the end user sees—it is a system-level recovery instruction that your application constructs and sends to the model on the user's behalf.

To implement this, maintain an ordered priority list for context sections in your application configuration. For example: [SYSTEM_INSTRUCTION, USER_QUERY, RECENT_MESSAGES, TOOL_OUTPUTS, RETRIEVED_DOCUMENTS, OLDER_MESSAGES, EXAMPLES]. When an overflow error fires, your harness removes the lowest-priority section and reconstructs the prompt. The retry template itself should receive the original task description, the error message, and the reduced context. Validate the model's response against your original output schema before returning it to the user. If the retry also fails with an overflow error, repeat the reduction cycle—removing the next lowest-priority section—up to a configurable maximum retry count (we recommend 3). After the final attempt, escalate: return a partial result with a clear truncation_notice field, log the full context for debugging, and optionally route to a larger-context model or a human reviewer.

Log every reduction step with the sections removed, the new token estimate, and whether the retry succeeded. This trace data is essential for tuning your priority ordering and identifying workflows that consistently overflow. Avoid the common failure mode of blindly truncating from the end of the prompt—this often removes the user's actual request or the output schema. Instead, use the priority list to protect high-value context. If your application handles regulated or high-stakes content, require a human to review any response produced after two or more reduction cycles before it reaches the end user.