Token-Limit Cutoff Detection and Recovery Prompt Template

This prompt is for API developers and production engineers who need to detect when a model output was truncated by the max_tokens parameter and request an intelligent continuation. The core job-to-be-done is recovering a complete, valid payload when the model stopped mid-generation due to a hard token limit, without discarding the partial work and restarting from scratch. The ideal user is someone running batch inference, non-streaming completions, or agent tool-call loops where a finish_reason: 'length' signal indicates the output is incomplete. Required context includes the original prompt, the partial output, the output schema or expected format, and the exact truncation point.

Use this prompt when you have a clear truncation signal (API finish_reason equals length or max_tokens), the partial output contains recoverable structure, and the cost of restarting generation exceeds the cost of a continuation request. This is a post-generation repair workflow, not a real-time streaming strategy. Do not use this prompt when the output is malformed for reasons other than token limits—such as schema violations, hallucinated fields, or encoding errors—which require different repair playbooks from the Output Repair and Validation pillar. Do not use this prompt when the partial output is so severely truncated that no meaningful structure remains; in those cases, restarting generation with a higher max_tokens value or chunking the task is more reliable. Avoid this approach in latency-sensitive streaming applications where a continuation request would introduce unacceptable delay.

Before deploying this prompt, verify that your application can reliably detect truncation. The OpenAI and Anthropic APIs both return an explicit finish_reason field. If you are using a local model or proxy that does not surface this signal, you must implement your own truncation detection by checking for unclosed brackets, braces, XML tags, or code blocks in the output. Wire the continuation request to preserve the original system prompt, temperature, and tool definitions to avoid context drift. Always validate the stitched output against the expected schema before accepting it. For high-risk domains such as healthcare or finance, flag stitched outputs for human review rather than silently accepting the repaired payload.

This prompt template is designed to be used inside a recovery handler when your application detects that a model response was truncated due to hitting the max_tokens limit. Instead of discarding the partial output and starting over—which wastes tokens and can lose valuable reasoning—this prompt instructs the model to analyze the cut-off point and generate only the missing continuation. The template uses square-bracket placeholders that your application must populate with the original request context, the truncated response, and any output schema constraints.

textCopy
You are a recovery assistant. Your only job is to continue a truncated AI response.

## ORIGINAL REQUEST
[ORIGINAL_USER_PROMPT]

## SYSTEM INSTRUCTIONS FROM ORIGINAL REQUEST
[ORIGINAL_SYSTEM_PROMPT]

## TRUNCATED RESPONSE (CUT OFF BY TOKEN LIMIT)
[TRUNCATED_OUTPUT]

## OUTPUT SCHEMA (IF APPLICABLE)
[OUTPUT_SCHEMA]

## TASK
1. Identify the exact point where the truncated response was cut off.
2. Determine what content, structure, or reasoning was likely intended to follow.
3. Generate ONLY the missing continuation. Do not repeat any content from the truncated response.
4. If the truncated response ends mid-word, mid-JSON field, or mid-code block, repair that boundary before continuing.
5. Preserve the original tone, format, and output schema exactly.
6. If the truncated response appears complete (false positive), respond with: [COMPLETE_MARKER].

## CONSTRAINTS
- Do not add introductory phrases like "Here is the continuation" or "Continuing from where it left off."
- Do not summarize or rephrase the truncated content.
- Output only the continuation text, ready to be concatenated directly to the truncated response.

To adapt this template for your application, replace each placeholder with the actual values from the original request. [ORIGINAL_USER_PROMPT] should contain the full user message that triggered the initial generation. [ORIGINAL_SYSTEM_PROMPT] should include the system instructions active during the original call—this is critical for tone and constraint preservation. [TRUNCATED_OUTPUT] is the partial response your application received before the token limit was hit. [OUTPUT_SCHEMA] should contain any JSON schema, XML DTD, or format specification the original output was expected to follow; if none exists, replace this with "No specific output schema required." The [COMPLETE_MARKER] token should be a unique string your application can detect to avoid unnecessary concatenation when the original response was not actually truncated.

Before deploying this prompt into production, wire it into a recovery harness that validates the concatenated output against your original schema and checks for boundary artifacts. Common failure modes include the model repeating the last few tokens of the truncated response (causing duplication at the seam), generating a continuation that drifts from the original output format, or failing to repair a mid-token cut. Your harness should trim overlapping tokens at the concatenation boundary and run the full output through the same validators used for non-truncated responses. For high-risk domains such as healthcare or finance, flag recovered outputs for human review rather than automatically surfacing them to users.

This prompt is designed to sit inside a post-generation recovery loop, not as a standalone utility. After your primary model call returns a finish_reason of length (or equivalent for your provider), you should immediately route the partial output and the original request context into this prompt. The harness must preserve the exact truncation point—do not trim or clean the partial output before passing it, as trailing whitespace, broken tokens, or mid-word cutoffs are the primary signals the recovery prompt uses to detect the boundary. Store the original max_tokens limit, the actual token count consumed, and the finish_reason in your trace metadata before initiating recovery.

Validation gates should run on both the partial input and the completed output. Before calling the recovery prompt, validate that the partial output is non-empty and that the truncation is genuine—a finish_reason of length with tokens remaining in your budget may indicate a different failure mode. After receiving the completed output, run your standard schema validators, parse checks, or content policies against the full result. If the completed output still fails validation, increment a retry counter and re-invoke the recovery prompt with the validator's error message appended to [CONSTRAINTS]. Set a hard retry limit (typically 2–3 attempts) before escalating to a human review queue or logging the partial output as unrecoverable. Log every recovery attempt with the attempt number, token counts, latency, and validation result for later analysis.

Model selection matters here. The recovery prompt works best with models that share the same tokenizer and context window as the original generation model—cross-model recovery can introduce boundary misalignment if tokenization differs. If you must use a different model for recovery, include the original model name in [CONTEXT] and test boundary detection accuracy across model pairs before deploying. For streaming applications, buffer the final chunk and check the stream's termination reason before assembling the partial output; never pass individual streaming chunks to the recovery prompt, as it expects the full partial payload. Wire the recovery prompt into your existing observability stack with a dedicated recovery_attempt span, and set alerts on recovery failure rates exceeding your team's threshold—sustained failures often indicate a prompt design problem upstream, not a recovery logic bug.