Budget-Aware Context Assembly Orchestration Prompt

This prompt is for AI engineering teams building production systems where the context window is a scarce and expensive resource. The job-to-be-done is real-time, programmatic assembly of a model request: you have a system message, user input, retrieved evidence, tool schemas, conversation history, and few-shot examples, but you cannot fit everything. This prompt acts as an orchestrator that produces a specific, token-counted assembly plan—deciding what to include in full, what to compress, and what to exclude—before you send the final prompt to inference. The ideal user is a developer or AI operator embedding this step into a prompt assembly pipeline, not an end-user chatting with a model.

Use this prompt when context assembly decisions are too complex for static truncation rules. For example, a RAG pipeline retrieving 50 passages needs to pack the top-k into a 4,000-token evidence block while preserving instruction fidelity. A simple FIFO drop or naive summarization will break answer quality. This prompt evaluates salience, deduplication, and instruction priority to produce a budget-compliant plan. It is also appropriate when you need a preflight compliance check: the output includes a budget_compliant boolean and a fallback_strategy field, allowing your application to block non-compliant requests or route them to a larger-context model. Integrate this into your request builder as a validation gate before the final inference call.

Do not use this prompt for simple, single-turn requests where the context fits comfortably within the model's limit. It adds latency and cost that are only justified when assembly logic is non-trivial. Avoid it for real-time chat where a user expects an immediate response; the orchestration step should be asynchronous or have a strict timeout. This prompt is not a substitute for proper retrieval ranking or embedding-based similarity—it operates on already-retrieved candidates. Finally, this prompt does not execute the assembly; it produces a plan. Your application code must parse the plan and construct the final prompt payload. Always validate the output against a JSON schema before acting on it, and log the assembly plan for observability.

The following prompt template is designed to be dropped into a prompt assembly pipeline. It takes a set of candidate context blocks—instructions, retrieved evidence, few-shot examples, tool schemas, conversation history—and produces a concrete assembly plan with section-level token allocations, inclusion decisions, and fallback strategies. Use it when your application must make real-time packing decisions under a strict context window budget.

textCopy
You are a context assembly orchestrator. Your job is to produce a token-aware assembly plan for a single model request given a set of candidate context blocks and a hard token budget.

## INPUTS
- Candidate context blocks with token counts:
[CANDIDATE_BLOCKS]
- Hard token budget (total tokens available for the assembled prompt):
[TOKEN_BUDGET]
- Priority ordering rules:
[PRIORITY_RULES]
- Output schema for the final model response:
[OUTPUT_SCHEMA]
- Risk level (low, medium, high):
[RISK_LEVEL]

## CONSTRAINTS
[CONSTRAINTS]

## TASK
1. Classify each candidate block as: REQUIRED, IMPORTANT, NICE_TO_HAVE, or EXCLUDABLE.
2. Allocate tokens to each block following the priority ordering rules.
3. If the total required tokens exceed the budget, apply compression strategies in this order:
   a. Summarize conversation history to essential turns only.
   b. Prune low-salience evidence passages.
   c. Reduce few-shot examples to the most representative subset.
   d. Compress tool schemas to essential parameters only.
4. If compression still exceeds budget, produce a fallback plan that preserves REQUIRED blocks and drops lower-priority blocks with explicit justification.
5. Reserve at least [RESERVED_TOKENS] tokens for the output schema and final model response.

## OUTPUT FORMAT
Return a JSON object with this exact structure:
{
  "plan_id": "string",
  "total_budget": number,
  "total_allocated": number,
  "reserved_for_output": number,
  "blocks": [
    {
      "block_id": "string",
      "category": "REQUIRED|IMPORTANT|NICE_TO_HAVE|EXCLUDABLE",
      "original_tokens": number,
      "allocated_tokens": number,
      "action": "INCLUDE_FULL|INCLUDE_COMPRESSED|EXCLUDE",
      "compression_method": "string|null",
      "justification": "string"
    }
  ],
  "compression_applied": boolean,
  "fallback_triggered": boolean,
  "fallback_plan": "string|null",
  "budget_compliance": {
    "within_budget": boolean,
    "overflow_tokens": number,
    "overflow_resolution": "string"
  }
}

## PREFLIGHT CHECKS
Before returning the plan, verify:
- Total allocated tokens + reserved tokens <= total budget.
- All REQUIRED blocks are included (full or compressed).
- No block exceeds its original token count unless explicitly justified.
- Fallback plan is present if and only if fallback_triggered is true.

Adapt this template by replacing the square-bracket placeholders with your runtime values. [CANDIDATE_BLOCKS] should be a JSON array of objects, each with a block_id, content summary, token_count, and source_type (e.g., system_instruction, retrieved_passage, few_shot_example, tool_schema, conversation_turn). [PRIORITY_RULES] defines the ordering—for example, "system instructions first, then user query, then evidence ranked by salience, then examples, then tool schemas." [CONSTRAINTS] can specify domain-specific rules such as "never drop safety policy instructions" or "always include at least 3 evidence passages for high-risk queries." [RESERVED_TOKENS] should account for the expected output length plus any fixed overhead from the output schema. For high-risk domains, set [RISK_LEVEL] to "high" and add a human review step before the assembled prompt is sent to the model.

The orchestration prompt is not a standalone tool; it is a decision engine that sits between your context retrieval system and your final inference call. In a production harness, you should call this prompt after you have assembled all candidate context blocks—instructions, retrieved passages, conversation history, tool schemas, and few-shot examples—but before you send the final request to the target model. The prompt receives a manifest of available context blocks with their token counts and a total budget, and it returns an assembly plan specifying what to include, compress, or exclude. Wire this as a synchronous preflight step in your inference pipeline, with a strict timeout (e.g., 2-5 seconds) to avoid adding unacceptable latency. If the orchestrator times out, fall back to a static packing strategy: include the system message and the most recent N turns of conversation, then fill remaining budget with top-ranked evidence until the limit is reached.

Validation is critical before acting on the assembly plan. After receiving the JSON output, run a budget compliance check: sum the allocated token counts for all included blocks and verify the total does not exceed the declared budget. If it does, reject the plan and either retry with an explicit overflow error message or fall back to a truncation-based strategy. Next, run a completeness check: confirm that non-negotiable blocks (e.g., safety policies, required tool schemas) are present in the included set. If any mandatory block is missing, flag the plan for human review or automatic correction. Log the full assembly plan, token allocations, and validation results to your observability platform with a trace ID that links the orchestration decision to the downstream inference call. This traceability is essential for debugging quality regressions—when an answer is poor, you need to know whether the orchestrator excluded critical evidence or compressed instructions too aggressively.

Model choice for the orchestrator itself matters. Because this prompt performs structured reasoning over token counts and priorities, a capable but cost-efficient model (e.g., GPT-4o-mini, Claude Haiku, or a fine-tuned small model) is usually sufficient. Avoid using your most expensive model for orchestration unless the assembly decisions are exceptionally high-stakes. Implement a retry policy with a maximum of two attempts: if the first call returns malformed JSON or fails validation, retry once with the validation errors injected into the prompt as feedback. If the second attempt also fails, log the failure, fall back to the static packing strategy, and increment a metric for orchestration failure rate. For high-risk domains such as healthcare or legal workflows, route any plan that excludes more than 20% of candidate evidence blocks to a human reviewer before proceeding. Finally, periodically run offline eval suites that compare assembly plans against human-annotated ideal packings, measuring both token efficiency and downstream answer quality to detect drift in the orchestrator's judgment over time.