Streaming Output Truncation Detection and Repair Prompt Template

This prompt is designed for API developers and streaming infrastructure engineers who need to recover from mid-stream output cutoffs caused by token limits, connection drops, or model early-stopping. The primary job is to detect incomplete sentences, unclosed structured fields (like JSON braces or XML tags), or truncated code blocks in a partial payload, and then either request a precise continuation or repair the output into a valid partial state that downstream parsers can handle without crashing. Use this when your application consumes streaming LLM responses and you cannot afford to discard partially delivered work—such as in real-time user interfaces, log processors, or agent execution traces where losing context mid-stream creates a broken user experience or corrupts a database record.

The prompt works best when you have access to the raw truncated text and the expected output schema or format contract. For example, if your system expects a complete JSON object with required fields, the prompt can identify that a closing brace is missing and the "summary" field was cut off after 12 characters, then either generate the missing fragment or produce a valid partial object with a "truncated": true flag. It is less effective when the truncation point is ambiguous—such as when a model stops mid-word in a free-text narrative with no structural markers—or when the missing content is semantically critical and cannot be safely inferred. In those cases, requesting a continuation from the original model with the truncated text as context is more reliable than attempting repair.

Do not use this prompt as a substitute for proper token budgeting, max_tokens configuration, or streaming buffer management. If you routinely hit token limits, fix the root cause by reducing context size, implementing prompt compression, or splitting work into smaller chunks. This prompt is a safety net, not a primary strategy. Additionally, avoid using it for outputs where truncation could introduce safety or compliance risks—such as incomplete medical instructions, partial legal clauses, or financial figures—without mandatory human review of the repaired output. For high-stakes domains, flag truncated outputs for escalation rather than attempting automated repair.

This template provides a ready-to-use prompt for detecting truncation in streaming LLM outputs and either requesting continuation or repairing the output to a valid partial state. It is designed to be called after a stream has ended unexpectedly—due to token limits, connection drops, or model early-stopping—and the raw output fails completeness checks. The prompt accepts the partial output, the expected schema or structure, and the stream's termination reason, then returns a structured assessment and repair action.

textCopy
You are an output repair agent for a streaming LLM system. Your job is to analyze a partial output that was cut off mid-stream and determine whether it can be repaired to a valid partial state or requires continuation.

## INPUT
- Partial Output: [PARTIAL_OUTPUT]
- Expected Output Schema or Structure Description: [OUTPUT_SCHEMA]
- Stream Termination Reason: [TERMINATION_REASON]
- Maximum Continuation Tokens Available: [MAX_TOKENS]

## TASK
1. Analyze the partial output for truncation indicators:
   - Incomplete sentences or paragraphs
   - Unclosed JSON objects, arrays, or strings
   - Unclosed XML tags or markdown blocks
   - Missing required fields per the expected schema
   - Trailing content that appears cut off mid-token

2. Classify the truncation severity:
   - `repairable`: The output can be closed to a valid partial state without adding new substantive content
   - `continuable`: The output requires additional generation to reach a valid stopping point
   - `unrecoverable`: The output is too corrupted to repair or continue meaningfully

3. If `repairable`, produce the repaired output by:
   - Closing unclosed structures (JSON braces, XML tags, code fences)
   - Truncating incomplete sentences at the last complete sentence boundary
   - Marking missing required fields as `null` with a `_truncated: true` annotation
   - Preserving all valid content without modification

4. If `continuable`, produce a continuation request that:
   - Specifies the exact text to continue from (last 50-100 characters of partial output)
   - Describes what structure or content must be completed
   - Includes any constraints to prevent repetition or hallucination

5. If `unrecoverable`, explain why and recommend fallback behavior.

## OUTPUT FORMAT
Return a JSON object with this exact structure:
{
  "classification": "repairable" | "continuable" | "unrecoverable",
  "truncation_indicators": ["list of specific indicators found"],
  "repair_action": {
    "repaired_output": "<repaired output if repairable, null otherwise>",
    "repairs_applied": ["list of specific repairs made"]
  },
  "continuation_request": {
    "continuation_prompt": "<continuation prompt if continuable, null otherwise>",
    "expected_completion": "<description of what must be completed>"
  },
  "fallback_recommendation": "<recommended fallback if unrecoverable>"
}

## CONSTRAINTS
- Do not invent content to fill gaps unless the classification is `repairable` and the repair is purely structural
- Do not modify any valid content from the partial output
- If continuing, the continuation prompt must include enough context to prevent topic drift
- Mark all repaired or incomplete fields explicitly

To adapt this template, replace the square-bracket placeholders with your runtime values. [PARTIAL_OUTPUT] should contain the raw truncated text exactly as received from the stream. [OUTPUT_SCHEMA] should describe the expected structure—this can be a JSON Schema, a natural language description of required fields, or a reference to your application's output contract. [TERMINATION_REASON] should indicate why the stream ended (e.g., token_limit, connection_closed, model_stopped). [MAX_TOKENS] constrains the continuation budget. For high-risk applications where the repaired output feeds directly into user-facing surfaces or databases, add a human review step before accepting the repair action. Test this prompt against known truncation patterns—mid-sentence cuts, unclosed JSON at various nesting depths, and streams that end exactly at a valid boundary—to ensure the classification logic is reliable before deploying to production.

The truncation detection and repair prompt is not a standalone utility—it is a recovery step inside a streaming output pipeline. Wire it after your chunk assembly layer and before your schema validator. When the assembled output fails a completeness check (unclosed JSON, truncated sentence, missing required fields), route the partial payload to this prompt along with the original generation context. The prompt returns either a repaired complete output or a structured continuation request that your application can send back to the model provider. Design the harness to distinguish between repairable truncation (the model can infer the missing content from context) and unrecoverable truncation (critical data was lost and must be regenerated).

Implement a truncation detector as a lightweight pre-check before invoking the LLM repair prompt. This detector should run deterministic rules: check for unclosed brackets, braces, or quotes in structured outputs; detect sentence fragments at the end of text outputs; verify that required schema fields are present and non-null. Only invoke the repair prompt when truncation is detected—avoid wasting tokens on complete outputs. The detector should classify truncation severity: minor (trailing punctuation or whitespace), moderate (incomplete final sentence or field), and severe (mid-structure cutoff with missing required data). For severe truncation, prefer requesting a continuation from the model with the original prompt and accumulated context rather than attempting repair. Log the truncation type, severity, and repair decision to your observability stack for later analysis of model behavior and token limit tuning.

Build a retry and fallback circuit around the repair step. If the repair prompt itself fails or returns an invalid output, retry once with explicit error context. If the continuation request to the model provider times out or returns another truncated response, fall back to serving the best available partial output with a truncation flag set in the response metadata. Your application layer should handle this flag gracefully—display a 'response truncated' indicator to users or queue the record for async repair. Set a hard limit on repair attempts (we recommend 2 retries maximum) to avoid latency spikes. For high-throughput systems, consider running truncation repair as an async sidecar process: serve the partial response immediately, then patch the complete response into your data store once repair succeeds.

Choose your model routing carefully. Truncation repair is a low-latency, high-accuracy task that benefits from fast, instruction-following models. Use a smaller, cheaper model for repair (GPT-4o-mini, Claude Haiku, or equivalent) rather than your primary generation model. This keeps repair costs low and latency predictable. However, for domain-specific content where the repair requires deep context understanding, route to the same model class used for generation. Implement a model fallback chain: try the fast repair model first, escalate to a more capable model if the repair output fails validation. Always pass the original generation prompt, the partial output, and the detected truncation point as context—never ask the repair model to guess what was being generated.

Wire validation gates after repair. The repaired output must pass the same schema validation, completeness checks, and safety filters as a normal complete response. If the repair introduces hallucinated content (detected by comparing pre-truncation and post-repair outputs), flag the response for human review rather than serving it automatically. For regulated or high-stakes domains, require human approval on all repaired outputs before they reach end users or downstream systems. Store the pre-repair partial output, the repair prompt, the repaired output, and the validation results in an audit log. This traceability is essential for debugging truncation patterns, tuning token limits, and demonstrating repair quality to stakeholders.