Output Size Boundary Enforcement Prompt

This prompt is a pre-generation guardrail for API platform engineers and infrastructure teams who need deterministic control over output size before a model is invoked. Its primary job is to act as a sizing estimator and boundary enforcer: given a user request, system context, and model constraints, it predicts whether the likely response will exceed a configurable [MAX_OUTPUT_TOKENS] limit. If the estimate exceeds the budget, the prompt produces a structured decision to either reject the request early with a clear reason, or propose a specific truncation strategy (e.g., 'summarize in half the tokens', 'omit code examples', 'respond only to the first two questions'). This prevents the default failure mode where a model hits its max_tokens parameter mid-generation, delivering a truncated, often unusable, response to the user while still incurring the full compute cost.

Use this prompt when you are building an API gateway, model router, or internal platform service that fronts multiple models with different context windows and pricing tiers. It is especially valuable in high-throughput, cost-sensitive environments where a single unconstrained request can waste tens of thousands of tokens. The prompt is designed to be model-agnostic; it reasons about token budgets conceptually rather than relying on a specific model's tokenizer. This makes it suitable for environments where you route between models like GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro, each with different effective context windows and per-token costs. You should wire this prompt into your request pipeline after retrieval augmentation and context assembly but before the final model invocation, so it can account for the full prompt payload when making its estimate.

Do not use this prompt as a substitute for proper max_tokens parameter configuration on your API calls. It is a pre-flight check, not a replacement for hard limits. It is also not suitable for real-time streaming use cases where the output size is genuinely unknown or where partial results are acceptable; in those scenarios, a streaming-aware boundary enforcement with chunk-level checks is more appropriate. Avoid using this prompt for trivial requests where the overhead of the estimation step exceeds the cost of the generation itself. The next section provides the copy-ready prompt template you can adapt with your own limits, output schemas, and truncation strategies.

This prompt template is designed to be placed as a final guard in your prompt assembly pipeline, immediately before the model generates a response. Its job is to analyze the combined input context and the requested output task, estimate whether the expected output would exceed a configurable token budget, and enforce a boundary decision. Use this when you cannot afford silent truncation, when downstream consumers have strict size limits, or when you need to choose between summarization, rejection, or model escalation before spending compute on a response that will be cut off.

codeCopy
SYSTEM: You are an output size boundary enforcement layer. Your only job is to analyze the request and decide whether the expected output will fit within the token budget. Do not generate the requested output itself.

INPUT_CONTEXT_LENGTH: [INPUT_TOKEN_COUNT]
MAX_OUTPUT_TOKENS: [MAX_OUTPUT_TOKENS]
MODEL_CONTEXT_LIMIT: [MODEL_CONTEXT_LIMIT]
REQUESTED_TASK: [TASK_DESCRIPTION]
USER_INPUT: [USER_INPUT]

INSTRUCTIONS:
1. Calculate the remaining context window: MODEL_CONTEXT_LIMIT - INPUT_CONTEXT_LENGTH.
2. Estimate the minimum tokens required to fulfill the REQUESTED_TASK given the USER_INPUT. Be conservative; if the task asks for a list of 100 items, assume at least 100 tokens per item.
3. If the estimated output tokens exceed MAX_OUTPUT_TOKENS or the remaining context window, classify the request as OVER_BUDGET.
4. If OVER_BUDGET, recommend one of the following strategies in the `strategy` field:
   - `reject`: The task cannot be reasonably completed within the budget. Provide a user-facing rejection message.
   - `truncate`: The task can be partially completed. Specify a safe output limit and a truncation instruction.
   - `summarize`: The full output should be compressed. Provide a summarization instruction.
   - `escalate`: The task requires a model with a larger context window. Recommend a fallback model.
5. If the request is WITHIN_BUDGET, set `action` to `proceed` and pass through the original task unchanged.

OUTPUT_SCHEMA:
{
  "decision": "WITHIN_BUDGET" | "OVER_BUDGET",
  "estimated_output_tokens": <integer>,
  "remaining_context_window": <integer>,
  "action": "proceed" | "reject" | "truncate" | "summarize" | "escalate",
  "strategy_details": {
    "safe_output_limit": <integer | null>,
    "truncation_instruction": "<string | null>",
    "rejection_message": "<string | null>",
    "fallback_model": "<string | null>"
  },
  "reasoning": "<brief explanation of the decision>"
}

To adapt this template, replace the square-bracket placeholders with values from your application context. [INPUT_TOKEN_COUNT] should be the actual token count of the assembled prompt, obtained from your tokenizer before invocation. [MAX_OUTPUT_TOKENS] is your application's hard limit, which may be stricter than the model's maximum. [MODEL_CONTEXT_LIMIT] is the model's total context window. [TASK_DESCRIPTION] should be a concise summary of what the user asked the model to do, which helps the boundary layer estimate output size without executing the full task. In streaming scenarios, run this check before opening the stream; if the decision is OVER_BUDGET with a reject action, return the rejection message immediately without streaming. For truncate actions, inject the truncation_instruction into the main prompt and set max_tokens to the safe_output_limit. Always log the decision and estimated_output_tokens fields for cost and latency observability. In high-stakes applications where rejection could frustrate users, pair this prompt with a human review queue for OVER_BUDGET decisions that fall within a configurable margin (e.g., within 10% of the budget).

The output size boundary enforcement prompt is designed to sit as a pre-generation guard in your model invocation pipeline, not as a post-hoc truncation tool. It should be called after the full prompt is assembled but before the final generation request is sent to the model. The prompt takes the composed system prompt, user input, retrieved context, and tool definitions, then estimates whether the expected output would exceed a configurable [MAX_OUTPUT_TOKENS] threshold. If the estimate indicates an overflow, the prompt returns a structured rejection with a recommended truncation strategy rather than allowing the model to generate a truncated, incomplete, or hallucinated response. This pre-generation check is critical for streaming endpoints where mid-stream truncation produces broken JSON, incomplete function calls, or sentences cut mid-word.

To wire this into an application, implement a pre-generation hook in your model gateway that: (1) calculates the current input token count using the model's tokenizer, (2) subtracts that count from the model's context window to determine available output capacity, (3) compares available capacity against [MAX_OUTPUT_TOKENS], and (4) if the available capacity is below the threshold, invokes this prompt with the full composed context and the estimated token budget. The prompt returns a JSON object with fields boundary_violated (boolean), estimated_output_tokens (integer), available_output_tokens (integer), truncation_strategy (string enum: reject, summarize_context, reduce_retrieved_chunks, split_request), and user_facing_message (string). Your gateway should parse this response and either proceed with generation or return the user-facing message. For streaming endpoints, cache this decision per request to avoid re-running the check on each chunk. Log every boundary violation with the request ID, estimated token counts, chosen strategy, and whether the user retried with a modified input—this data is essential for tuning your [MAX_OUTPUT_TOKENS] threshold and identifying patterns in oversized requests.

Validation and retry logic must be explicit. If the prompt returns malformed JSON or a missing boundary_violated field, your harness should default to a safe rejection with a generic message rather than proceeding blindly. Implement a circuit breaker: if the boundary check prompt itself fails more than 3 times in a 60-second window, fail open with a warning log and allow generation to proceed, but flag the incident for review. For high-throughput systems, consider caching boundary check results for identical composed prompts using a hash of the full prompt text as the cache key. Avoid caching when user-specific data is present unless your cache is tenant-isolated. The most common production failure mode is tokenizer mismatch: if your gateway uses a different tokenizer than the model's native tokenizer, the estimated counts will be wrong. Always use the model's official tokenizer for count calculations, and add a 5% safety margin to your [MAX_OUTPUT_TOKENS] threshold to account for tokenization variance.

When deploying this prompt in a multi-model routing setup, the boundary check must be model-aware. Different models have different context windows, tokenizers, and output behaviors. Store per-model [MAX_OUTPUT_TOKENS] thresholds and context window limits in your configuration, and inject the correct values into the prompt at runtime. For RAG systems, the boundary check should run after retrieval but before generation, and the truncation_strategy field should guide whether to reduce the number of retrieved chunks, summarize them, or split the request into multiple sub-requests. If the strategy is split_request, your harness must have a downstream workflow that can handle multi-turn or multi-request decomposition. Never silently truncate retrieved context without logging what was removed—this creates a hallucination risk where the model answers from incomplete evidence. The next step after implementing this harness is to build eval cases that test boundary-near inputs, inputs that exceed the limit by small and large margins, and inputs where the boundary check itself times out or returns invalid JSON.