Token Budget Exhaustion Fallback Prompt

This prompt is for agent runtime developers and operators who need a deterministic, programmatic fallback when an agent's context window is nearly full mid-execution. The job-to-be-done is to produce a compressed state summary, a prioritized list of remaining steps, and a binary decision: continue with summarization or abort with partial results. The ideal user is an engineering lead integrating this into an agent harness that tracks token consumption in real time and can inject this prompt as a preflight check before the next tool call or reasoning step. Required context includes the current plan, completed steps with outputs, remaining steps, and the exact token budget remaining. Do not use this prompt for general context management, initial plan generation, or when the agent has not yet consumed 70%+ of its budget; it is a last-resort circuit breaker, not a routine compression tool.

This prompt is designed to be wired into a token-budget monitor that fires when remaining_tokens < (estimated_next_step_tokens * 2). The harness should provide the model with a precise budget ceiling for the compressed output itself, typically 15-25% of the original remaining budget, to prevent the fallback from consuming the very resource it is trying to preserve. The output must be machine-parseable JSON with a decision field (continue or abort), a compressed_state object, and a prioritized_remaining_steps array. Validation must check that the compressed state is strictly smaller than the original state representation and that no completed steps are re-listed as remaining. Human review is not required for the fallback decision itself, but the harness should log the full pre-compression state and the compressed output for post-hoc audit. If the model fails to produce valid JSON or exceeds the output token budget, the harness must default to abort with a structured partial-results handoff.

Do not use this prompt when the agent is operating under a stateless retry model where a fresh context is cheaper than compression, or when the remaining steps include irreversible external actions that require full context fidelity. The primary failure mode is over-compression that drops critical constraints or dependency information, causing the agent to execute remaining steps with corrupted state. Mitigate this by requiring the compressed state to include an explicit lost_detail_flags array that lists any information the model knowingly omitted. Next, wire the output into your agent's execution loop with a post-compression validation step that diffs the original and compressed task lists for completeness. If the decision is continue, the agent should resume with the compressed state as its new working memory and a reduced token budget. If abort, the harness must assemble a user-facing partial completion summary using the structured output from this prompt.

The following prompt template is designed to be injected into an agent's execution loop when a token budget monitor detects that remaining context space is critically low. It forces the model to produce a structured fallback decision—either a compressed continuation plan or a partial abort with a handoff summary—before the context window overflows and loses state silently. Use this template as the final pre-exhaustion checkpoint in your agent harness.

textCopy
SYSTEM: You are an agent execution monitor. Your context window is nearly full. You must produce a structured fallback decision immediately. Do not continue executing tasks. Do not call tools. Output only the decision object.

CURRENT STATE:
- Completed steps: [COMPLETED_STEPS]
- Remaining steps with priorities: [REMAINING_STEPS]
- Active tool results pending: [PENDING_RESULTS]
- Original goal: [ORIGINAL_GOAL]
- Tokens used: [TOKENS_USED] / [TOKEN_LIMIT]
- Tokens remaining: [TOKENS_REMAINING]

CONSTRAINTS:
- [CONSTRAINTS]

DECISION RULES:
1. If the highest-priority remaining step can be completed with a compressed summary of prior state, output a CONTINUE decision with a compressed state summary and the single next step.
2. If multiple high-priority steps remain and compression would lose critical context, output an ABORT decision with a partial results summary and handoff instructions.
3. If any step involves [IRREVERSIBLE_ACTION_TYPES], prefer ABORT unless the step is already in progress and can be safely completed.

OUTPUT SCHEMA:
{
  "decision": "CONTINUE" | "ABORT",
  "compressed_state_summary": "string (max 500 tokens, only if CONTINUE)",
  "next_step": { "step_id": "string", "action": "string", "tool": "string", "args": {} },
  "partial_results_summary": "string (only if ABORT)",
  "handoff_instructions": "string (only if ABORT)",
  "reasoning": "string (brief justification for the decision)"
}

RISK LEVEL: [RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with values from your agent runtime. [COMPLETED_STEPS] and [REMAINING_STEPS] should be structured lists with step IDs, descriptions, and priority scores. [IRREVERSIBLE_ACTION_TYPES] is a list of action categories your system considers too risky to compress or continue without full context—such as database writes, external API mutations, or user-facing messages. [RISK_LEVEL] should be set to HIGH if any remaining step involves irreversible actions, and LOW otherwise, which can be used by downstream validation to decide whether a human review is required before acting on the ABORT or CONTINUE decision.

After copying this template, wire it into your agent harness so it is only invoked when [TOKENS_REMAINING] drops below a threshold you define (typically 15-20% of the total context window). Do not call this prompt on every step—it is a circuit breaker, not a planning prompt. Validate the output against the schema before acting on it, and log every invocation with the token state at the time of the decision for postmortem analysis.

This prompt is designed to be invoked by an agent runtime monitor, not by the agent's primary reasoning loop. The runtime must track cumulative token consumption across all LLM calls, tool outputs, and retrieved context within a session. When the remaining budget drops below a configured threshold (e.g., 15% of the model's context window), the runtime should inject this prompt as a system-level interrupt, passing the current plan state, completed steps, and remaining tasks. The prompt is not a suggestion—it is a forced decision point that produces a compressed state summary and an explicit continue/abort directive.

The implementation harness requires three components. First, a token budget tracker that sums prompt_tokens and completion_tokens from each API response, adds estimated overhead for tool call schemas and retrieval chunks, and compares the running total against a configurable max_budget (typically 80-90% of the model's context limit to leave room for the fallback prompt itself and a final response). Second, a state serializer that converts the agent's internal execution state—completed steps with outputs, pending steps with dependencies, tool call history, and any accumulated evidence—into a structured [CURRENT_STATE] block for the prompt. Third, a decision validator that parses the model's output, confirms it contains a valid decision field (continue_summarized or abort_partial), and enforces the directive: if abort_partial, the runtime must stop further LLM calls and return the partial results to the caller; if continue_summarized, the runtime must replace the full conversation history with the compressed summary before resuming execution.

For production safety, implement these guardrails. Budget threshold hysteresis: don't re-trigger the fallback prompt immediately after a continue_summarized decision—wait until at least 20% of the remaining budget is consumed or a minimum number of steps complete. Output schema enforcement: use structured output mode (JSON mode or function calling) to parse the compressed_state, prioritized_remaining_steps, and decision fields; if parsing fails, default to abort_partial with a logged error rather than risking unbounded execution. Logging and observability: record every fallback trigger event with the pre-compression token count, the compressed state size, the decision, and the post-compression token count to measure compression ratio and detect budget tracking drift. Human review for high-stakes workflows: if the agent is operating on mutable resources (database writes, API mutations, file system changes), route abort_partial decisions to a review queue with the partial results and remaining steps rather than silently discarding incomplete work.

Model choice matters here. This prompt requires strong instruction-following and structured output discipline. Use models with native JSON mode support (GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro) and avoid models under ~7B parameters that may struggle with the dual task of compression and prioritization. If using a smaller model for cost reasons, split the prompt into two sequential calls: first compression, then decision, validating the compression output before the decision step. Do not use this prompt with streaming-only endpoints that cannot guarantee complete structured output.

Before deploying, test the harness with simulated budget exhaustion scenarios. Create test cases where the agent is 50%, 75%, and 95% through a known task list when the budget threshold triggers. Verify that continue_summarized decisions produce compressed states that preserve critical context (task dependencies, tool outputs needed for remaining steps) and that abort_partial decisions correctly halt execution and return honest partial results. Measure compression ratio (pre-compression tokens / post-compression tokens) and set a minimum acceptable threshold—if the model fails to compress below 50% of the original state size, the fallback may not buy enough headroom to be useful, and you should consider architectural changes like external state storage or earlier checkpointing.