Context Window Overflow Risk Assessment Prompt Template

This prompt is for agent runtime developers who need to prevent context window overflows in long-running, multi-step agent workflows. Before executing a plan, you feed the plan steps, tool output schemas, and model context limit into this prompt. The model returns a step-by-step token consumption estimate, identifies the exact point where accumulated context would exceed the limit, and recommends where to insert checkpoint-and-summarize operations. Use this prompt during the plan review phase, after plan generation but before execution, to avoid mid-run failures that lose state and waste compute.

The ideal user is an AI engineer or platform developer building an agent orchestrator that manages autonomous workflows spanning dozens of tool calls. You should have access to the generated plan, the tool manifest with output schemas, and the target model's context window specification. The prompt works best when you can provide realistic estimates of tool output sizes—if every tool returns unbounded text, the assessment will be overly conservative. Wire this into your pre-execution validation pipeline so that plans exceeding the context budget are flagged before any side effects occur. For high-risk production systems, pair this assessment with a human approval gate: if the model recommends more than two checkpoint insertions, route the plan for manual review before execution.

Do not use this prompt for single-turn completions, simple RAG chains, or workflows with fewer than five steps where overflow risk is negligible. In those cases, the overhead of running a separate assessment prompt exceeds the value. Similarly, if your agent runtime already implements sliding-window context management or automatic summarization at fixed intervals, this prompt adds redundant analysis. The assessment is most valuable when you are deploying a new agent workflow, changing the underlying model, or modifying tool output schemas—any scenario where the context consumption profile is unknown. After running the assessment, store the results alongside the plan version so you can compare against actual token usage in production traces and refine your estimation heuristics over time.

This prompt template is designed to be dropped into an agent planning module before execution begins. It takes a generated plan, the model's context window limit, and an estimate of tokens consumed by system prompts and tool definitions. The model then analyzes each step's expected token accumulation, identifies the exact points where the context window would overflow, and recommends specific checkpoint or summarization insertion points. Use this when deploying agents that may run for dozens or hundreds of steps, where silent context truncation would cause the agent to lose critical state and produce corrupted outputs.

textCopy
You are a context window risk assessor for an agent runtime. Your job is to analyze a multi-step execution plan and predict where the model's context window will overflow, causing silent truncation and state loss.

## INPUTS

[PLAN_STEPS]
(A JSON array of plan step objects. Each step must include: step_id, description, expected_output_tokens, and any tool_calls with their expected input/output token sizes.)

[MODEL_CONTEXT_LIMIT]
(The maximum context window size in tokens for the target model, e.g., 128000 for GPT-4 Turbo or 200000 for Claude 3.5 Sonnet.)

[SYSTEM_PROMPT_TOKEN_COUNT]
(The number of tokens consumed by the system prompt, tool definitions, and any fixed prefix that persists across all turns.)

[ACCUMULATION_STRATEGY]
(How context accumulates: "full_history" if all prior messages are retained, "summarized" if a running summary replaces older turns, or "custom" with a description of the retention policy.)

[SAFETY_MARGIN_PERCENT]
(The percentage of the context window to reserve for output generation and unexpected growth. Typically 10-20%.)

## TASK

1. Calculate the cumulative token consumption at each step, starting from [SYSTEM_PROMPT_TOKEN_COUNT] as the baseline.
2. For each step, add the step's expected input tokens (prior context) and expected output tokens.
3. Identify the first step where cumulative tokens exceed ([MODEL_CONTEXT_LIMIT] * (1 - [SAFETY_MARGIN_PERCENT]/100)).
4. For each overflow point, recommend a specific intervention: checkpoint insertion (save state and reset context), summarization (compress prior steps), or plan restructuring (reorder steps to reduce peak context).
5. Flag any steps that individually exceed the context limit even with no prior context.

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "model_context_limit": number,
  "effective_limit": number,
  "baseline_tokens": number,
  "overflow_points": [
    {
      "step_id": string,
      "cumulative_tokens_before_step": number,
      "step_expected_tokens": number,
      "cumulative_tokens_after_step": number,
      "overflow_severity": "warning" | "critical",
      "recommended_intervention": "checkpoint" | "summarize" | "restructure" | "split_step",
      "intervention_detail": string
    }
  ],
  "steps_exceeding_individual_limit": [string],
  "recommended_checkpoint_steps": [string],
  "overall_risk_level": "low" | "medium" | "high" | "critical"
}

## CONSTRAINTS

- Assume tool call responses are included in context unless [ACCUMULATION_STRATEGY] specifies otherwise.
- If a step's expected output tokens are unknown, use the median output size from similar steps or flag as "uncertain" with a worst-case estimate.
- Do not recommend checkpoint insertion at every step; only where overflow is predicted.
- If the plan has no overflow risk, return an empty overflow_points array and overall_risk_level "low".
- Flag steps that rely on context that would be lost after a checkpoint, so the operator can decide whether to preserve that context explicitly.

Adaptation guidance: Replace each square-bracket placeholder with data from your agent runtime. The [PLAN_STEPS] input should come from your plan generation module, with token estimates derived from your model's tokenizer or from empirical averages per step type. If you lack per-step token estimates, add a pre-processing step that samples historical step outputs to build a token budget model. The [ACCUMULATION_STRATEGY] field is critical: if your runtime already uses sliding window summarization, the overflow points will shift dramatically compared to full-history retention. Test this prompt with both synthetic plans (known to overflow) and real production plans to calibrate the severity thresholds. For high-risk workflows where context loss could cause data corruption or compliance violations, route critical severity outputs to a human reviewer before the agent proceeds.

This prompt is designed to be called by an agent orchestrator or planning module before a long-running plan begins execution. It should not be a one-off manual check. Integrate it as a synchronous pre-flight gate that receives the proposed plan, the model's context window limit, and the current tokenizer's counting function. The output is a structured risk report that your runtime can parse to decide whether to proceed, insert checkpoints, or reject the plan. The prompt is most effective when it has access to accurate token counts per step, not rough estimates, so wire it to your production tokenizer (e.g., tiktoken for OpenAI models, or the model's native tokenizer via API).

Wiring the prompt into an agent runtime: The orchestrator should construct the prompt by injecting the full plan object, the model identifier and its context limit, and a pre-computed token budget breakdown per step. Use a structured output format (JSON) with a strict schema so the response can be parsed programmatically. On receiving the response, validate that the overflow_points array contains valid step indices and that recommended_checkpoint_steps are within the plan's step range. If the overflow_risk field is high and no checkpoints are recommended, treat this as a malformed response and retry once with a stricter constraint instruction. Log the full assessment to your observability platform, attaching the plan ID, model, and token budget for post-mortem analysis. If the assessment recommends checkpoints, your runtime should automatically insert state-serialization steps at the specified indices before execution begins. For plans where overflow is unavoidable even with checkpoints (e.g., a single step exceeds the context limit), escalate to a human operator or fall back to a model with a larger context window.

Validation and safety checks: Before trusting the assessment, confirm that the reported cumulative token counts align with your own tokenizer's output for the plan text. A mismatch of more than 10% should trigger a re-run with more explicit counting instructions. For high-risk workflows where context overflow could cause silent data loss or incorrect agent behavior, require a human to approve any plan flagged as overflow_risk: high before execution proceeds. Store the assessment alongside the plan version so that if the plan is reused or replayed, the overflow analysis is available without re-computation. Avoid using this prompt on plans that have already been compressed or summarized—run it on the original, unoptimized plan to get a true worst-case risk picture.