Context Overflow Graceful Degradation Prompt Template

This prompt is designed for product managers and UX engineers building user-facing AI features that operate on variable-length source material. The core job-to-be-done is to produce a model response that gracefully degrades when the provided context exceeds the model's effective processing window, rather than failing silently, hallucinating, or truncating mid-sentence. The ideal user is a product builder who needs the AI to explicitly communicate its limitations to the end-user while still delivering the maximum useful content from the portion of context it could process. Required context includes the full source text, the user's original query, and a clear definition of what 'useful content' means for this specific feature (e.g., a summary, a list of key entities, or an answer to a specific question).

Do not use this prompt when you can solve the problem architecturally through chunking strategies, sliding window summaries, or retrieval-augmented generation (RAG) that pre-selects relevant context. This prompt is a last-mile user-experience safeguard, not a primary context management strategy. It is appropriate when the context size is unpredictable, when you cannot modify the upstream context assembly pipeline, or when you need a safety net for edge cases where even your best context budgeting fails. It is also inappropriate for backend batch processing where no human reads the output; in those cases, use a structured truncation recovery prompt that triggers an automated continuation. For high-stakes domains like healthcare or legal review, this prompt must be paired with a human review step and should never be the sole mechanism for handling incomplete analysis.

Before implementing this prompt, define your evaluation criteria for 'graceful degradation.' A good degradation response should: (1) explicitly state that the context was too large to process fully, (2) indicate what portion was processed, (3) deliver the best possible output from the processed portion, and (4) suggest a path forward for the user (e.g., 'upload a shorter document' or 'ask a more specific question'). Test this prompt against documents of varying lengths, including those just below and just above your model's effective context limit. Measure whether users can distinguish a degraded response from a complete one and whether they can act on the partial information provided. Avoid the failure mode where the model apologizes profusely but delivers no actionable content—the prompt must balance transparency with utility.

This template is designed for product managers and engineers building user-facing AI features where context overflow is a known failure mode. Instead of crashing or returning an error, the model produces a partial, high-quality response that explicitly flags what it could not process and why. The prompt uses square-bracket placeholders so you can wire it into any application harness. Replace each placeholder with your actual task instructions, constraints, and output schema before use.

textCopy
You are an AI assistant operating under a strict context window limit. Your current context window is [MAX_TOKENS] tokens, and you estimate that the provided input consumes approximately [ESTIMATED_INPUT_TOKENS] tokens, leaving roughly [REMAINING_TOKENS] tokens for your response.

## TASK
[ORIGINAL_TASK_INSTRUCTION]

## INPUT
[FULL_INPUT_TEXT]

## CONSTRAINTS
[CONSTRAINTS]

## OUTPUT SCHEMA
[OUTPUT_SCHEMA]

## DEGRADATION RULES
If you detect that the input exceeds your available context or that you cannot complete the full task within the remaining token budget, follow these rules in order:

1. **Prioritize critical content.** Identify the most important sections of the input based on [PRIORITY_CRITERIA] and process those first.
2. **Produce a partial result.** Generate as much of the requested output as possible within the token budget, following the output schema for the completed portions.
3. **Mark incomplete sections.** For any part of the task you cannot complete, insert the marker `[TRUNCATED: reason]` where `reason` is one of: `context_overflow`, `token_budget_exhausted`, or `input_incomplete`.
4. **Add a limitations summary.** At the end of your response, append a `## LIMITATIONS` section that lists:
   - What was processed successfully
   - What was not processed and why
   - What the user should provide or adjust to get a complete result (e.g., shorter input, specific section to prioritize, higher token limit)
5. **Never fabricate content.** If you cannot process a section, do not guess or hallucinate. State that the section was not processed.

## EXAMPLES
[FEW_SHOT_EXAMPLES_OF_GRACEFUL_DEGRADATION]

## RISK LEVEL
[RISK_LEVEL: low | medium | high | critical]

If RISK_LEVEL is high or critical, append `[REQUIRES_HUMAN_REVIEW: true]` to the limitations summary and route the partial output to [HUMAN_REVIEW_QUEUE].

To adapt this template, start by defining your priority criteria explicitly. For a contract review task, priority might mean clauses with monetary values or termination rights. For a research synthesis task, priority might mean findings with statistical evidence. The degradation rules force the model to be honest about its limitations rather than silently dropping content. Wire the [TRUNCATED: reason] markers into your application's post-processing logic so your UI can render partial results with clear visual indicators of incompleteness. For high-risk domains, always set RISK_LEVEL to high or critical and ensure the human review queue is operational before deploying.

The Context Overflow Graceful Degradation prompt is designed to sit behind a user-facing feature where the system detects that the available context window is insufficient for a complete, high-confidence answer. Instead of silently truncating or returning an error, the application should invoke this prompt to produce a partial, useful output that explicitly communicates its limitations. The harness must first detect the overflow condition—typically by comparing the estimated token count of the assembled prompt (system instructions, retrieved context, conversation history, user input, and expected output buffer) against the model's context limit. When the estimated tokens exceed a configured threshold (e.g., 90% of the context window), the harness should route the request to this degradation prompt rather than the standard completion prompt.

Wire the prompt into your application as a fallback path in the model invocation layer. Before calling the primary model, run a token estimation function (using the target model's tokenizer or a conservative heuristic like character_count / 4). If the estimated total exceeds [MAX_CONTEXT_TOKENS] * [OVERFLOW_THRESHOLD], construct the degradation prompt with the original [USER_REQUEST], a compressed or truncated version of the available [CONTEXT], and the [CONSTRAINTS] that define what constitutes an acceptable partial answer. The model response should be parsed for the structured output defined in the prompt template—typically a JSON object with fields like partial_answer, completeness_assessment, limitations, and suggested_next_steps. Validate that the response contains all required fields and that the completeness_assessment is one of the expected enum values before surfacing it to the user.

Implement a retry budget specifically for degradation responses. If the model returns a malformed JSON object or fails to include the required limitation disclosure, retry up to [MAX_DEGRADATION_RETRIES] times with an appended instruction emphasizing the output schema. Log every degradation event with the estimated token count, the threshold that triggered it, the model's response, and whether the user accepted the partial answer or requested escalation. For high-stakes domains, route degradation responses through a human review queue before displaying them to end users. Avoid the common failure mode of applying degradation logic to every request—reserve it for cases where the context genuinely exceeds limits, and never degrade a request that fits comfortably within the window.

Model choice matters here. Models with larger context windows may reduce the frequency of degradation triggers, but they don't eliminate the need for the harness. Even with a 200K-token window, long-running agent sessions or document-heavy RAG workflows can hit limits. Test your degradation prompt across the models you support in production, as smaller or older models may struggle to produce well-structured partial answers under the same instructions. Pair this prompt with a context compression or summarization step before invoking degradation when possible—compress the available context first, then apply degradation only if the compressed context still exceeds the threshold. The next step after implementing this harness is to build monitoring dashboards that track degradation frequency, user satisfaction scores on partial answers, and escalation rates so you can tune thresholds and improve the degradation instructions over time.