Turn History Budget Allocation Prompt Template

Use the Turn History Budget Allocation Prompt when your chat, copilot, or agent system must programmatically divide a finite context window across competing input categories—system instructions, conversation history, retrieved evidence, and tool outputs—before making a model call. This is an upstream architectural decision prompt, not a content pruning or summarization prompt. Its job is to produce a structured allocation plan with explicit token budgets, priority ordering, and overflow handling rules. The plan answers 'how much space does each context category get?' before any downstream process selects, compresses, or drops specific turns or passages. This separation of concerns prevents the common failure mode where a summarizer or pruner consumes too much budget on one category and starves another of essential context.

This prompt is designed for production AI engineers who are embedding context window economics directly into their inference pipeline. Ideal deployment scenarios include: multi-turn RAG chat systems where retrieved evidence competes with conversation history for space; agent loops where tool outputs accumulate rapidly and threaten to displace system instructions; and cost-sensitive deployments where per-request token budgets are strict and must be enforced deterministically. The prompt expects structured inputs describing the available context categories, their minimum and maximum budget constraints, priority weights, and any hard retention requirements. It outputs a machine-readable allocation plan that can be consumed by downstream context assembly logic. Do not use this prompt when you need actual content selection, turn-level retention scoring, or summarization—those are separate downstream prompts that consume the budget plan this prompt produces. Also avoid this prompt for single-turn, stateless requests where context budgeting is trivial.

Before integrating this prompt, ensure you have instrumentation that reports current token counts per context category. The prompt's allocation decisions are only as good as the budget telemetry fed into it. Wire the output allocation plan into your context assembly harness as a set of hard limits: if the plan allocates 1200 tokens to retrieved evidence, your evidence packing step must not exceed that ceiling. Log allocation decisions alongside actual usage for observability. When the prompt's allocation plan results in degraded downstream task performance, investigate whether your priority weights accurately reflect the true information value of each context category for your specific use case. The next step after reading this section is to review the prompt template and adapt the input schema to match your system's actual context categories and budget constraints.

This prompt template is the core of your context budget allocation system. It instructs the model to act as a context window planner, producing a structured allocation plan that respects hard token limits, priority ordering, and overflow handling rules. The template uses square-bracket placeholders that you must replace with your actual values before assembly. The output is a machine-readable JSON schema designed to be consumed by your context assembly pipeline, not just read by a human.

textCopy
SYSTEM: You are a context window allocation planner. Your job is to produce a token allocation plan for a chat assistant's next request. You will receive the current conversation state, available evidence, pending tool outputs, and a hard token budget. You must allocate tokens across four categories: system instructions, conversation history, retrieved evidence, and tool outputs. Follow the priority order and overflow rules strictly. Do not hallucinate content. Output only valid JSON matching the schema.

USER:
<BUDGET>
Total Token Budget: [MAX_TOTAL_TOKENS]
System Instruction Reserve: [SYSTEM_RESERVE_TOKENS]
</BUDGET>

<PRIORITY_ORDER>
1. System Instructions (immutable, up to reserve limit)
2. Unresolved Questions and Pending Actions (from history)
3. User Corrections (from history)
4. Active Task Context (from history)
5. Retrieved Evidence (ranked by relevance)
6. Tool Outputs (ranked by recency and action-criticality)
7. Remaining Conversation History (most recent first)
</PRIORITY_ORDER>

<OVERFLOW_RULES>
- If budget is exceeded after allocating all Priority 1-3 items, emit a BUDGET_EXCEEDED warning and truncate Priority 4-7 to fit.
- Never truncate system instructions.
- Never drop unresolved questions or user corrections.
- When truncating history, drop the oldest, lowest-priority turns first.
- When truncating evidence, drop the lowest-ranked chunks first.
- When truncating tool outputs, drop the oldest, least action-critical outputs first.
</OVERFLOW_RULES>

<CONVERSATION_HISTORY>
[TURN_HISTORY]
</CONVERSATION_HISTORY>

<RETRIEVED_EVIDENCE>
[EVIDENCE_CHUNKS]
</RETRIEVED_EVIDENCE>

<TOOL_OUTPUTS>
[TOOL_RESULTS]
</TOOL_OUTPUTS>

<OUTPUT_SCHEMA>
{
  "plan": {
    "total_budget": [MAX_TOTAL_TOKENS],
    "allocated": {
      "system_instructions": {
        "tokens": 0,
        "status": "ALLOCATED"
      },
      "conversation_history": {
        "tokens": 0,
        "turns_included": [],
        "turns_excluded": [],
        "status": "ALLOCATED"
      },
      "retrieved_evidence": {
        "tokens": 0,
        "chunks_included": [],
        "chunks_excluded": [],
        "status": "ALLOCATED"
      },
      "tool_outputs": {
        "tokens": 0,
        "outputs_included": [],
        "outputs_excluded": [],
        "status": "ALLOCATED"
      }
    },
    "total_used": 0,
    "remaining": 0,
    "overflow": false,
    "warnings": []
  }
}
</OUTPUT_SCHEMA>

Generate the allocation plan.

To adapt this template, replace [MAX_TOTAL_TOKENS] with your model's context limit minus any overhead for the output itself. Replace [SYSTEM_RESERVE_TOKENS] with the exact token count of your system prompt. The [TURN_HISTORY] placeholder expects a structured representation of conversation turns, each with a turn ID, role, content, and metadata like is_correction or contains_unresolved_question. [EVIDENCE_CHUNKS] should be a list of retrieved passages with relevance scores and chunk IDs. [TOOL_RESULTS] should include tool call IDs, timestamps, and an action_critical flag. After receiving the plan, your application layer must validate that the total_used does not exceed the budget and that no Priority 1-3 items appear in exclusion lists. If the plan is invalid, re-prompt with the validation errors included as additional context.

The Turn History Budget Allocation Prompt is not a standalone prompt—it is a pre-processing step that runs before your main assistant prompt assembles the final context. In production, this prompt should be called inside a context assembly pipeline that receives raw conversation history, system instructions, retrieved evidence, and tool outputs, then returns a token-allocated, priority-ordered context plan. The plan output (typically JSON) is consumed by your application layer to construct the actual model request, not passed directly to the chat model as a user message.

Pipeline integration pattern: 1) Collect all candidate context blocks with their type tags (system_instruction, history_turn, retrieved_evidence, tool_output). 2) Call the budget allocation prompt with the candidate blocks, target token budget, and priority configuration. 3) Parse the JSON allocation plan. 4) Validate that total allocated tokens ≤ budget, all required system instructions are included, and no critical safety blocks were dropped. 5) Assemble the final prompt by concatenating blocks in the specified order. 6) Log the allocation plan, budget utilization, and any overflow items for observability. 7) If validation fails, retry with explicit error feedback or fall back to a safe default allocation (system instructions first, then most recent N turns).

Model choice and latency: Use a fast, cost-efficient model for this allocation step—GPT-4o-mini, Claude Haiku, or a fine-tuned small model work well. The allocation prompt is a deterministic planning task, not a creative generation task. Cache the system prompt prefix to reduce per-call cost. If your pipeline runs this prompt on every turn, budget for ~200-500 input tokens and ~150-300 output tokens per allocation call. For high-throughput systems, consider running allocation every K turns rather than every turn, or triggering re-allocation only when new evidence or tool outputs enter the context. Do not use this prompt inside the main assistant's context window—it runs as a separate, isolated call to avoid contaminating the assistant's working memory.

Validation and failure modes: The most common failure is the model allocating tokens that exceed the budget due to arithmetic errors. Implement a hard post-processing check: if sum(allocated_tokens) > budget, truncate the lowest-priority blocks until the budget is satisfied, then log the discrepancy. Another failure mode is the model dropping system instructions or safety policies—validate that all blocks tagged priority: critical appear in the allocation plan. If the model produces malformed JSON, retry once with a stricter schema instruction, then fall back to a rule-based allocation (system instructions first, then history in reverse chronological order, then evidence, then tool outputs, each capped at their proportional budget share).

Observability and tuning: Log every allocation decision: budget requested, budget used, blocks included, blocks overflowed, and the priority scores assigned. This telemetry lets you tune budget thresholds, identify which context types are consistently overflowed, and measure whether allocation quality correlates with downstream task success. Pair this prompt with an evaluation harness that compares allocation plans against human-annotated ideal allocations on a golden dataset of conversations. When you update the allocation prompt, run the eval suite to catch regressions before they degrade your main assistant's performance under budget pressure.