Tool Output Truncation Handling Prompt Template

This prompt is for AI engineers and platform developers who are building systems where large language models consume tool outputs that can exceed the model's context window. The core job-to-be-done is to produce a prioritized, loss-aware summary of a truncated tool result so the model can still generate a useful response without hallucinating missing information. The ideal user is someone integrating APIs, databases, or search systems where payload sizes are unpredictable and truncation is a runtime reality, not a design choice. Required context includes the raw truncated output, the user's original request, and any known truncation metadata such as byte limits or record counts.

Use this prompt when you cannot guarantee that tool outputs will fit within the model's context budget and you need a structured fallback that explicitly marks what was lost. This is common with log analysis tools, large database result sets, verbose API responses, or document retrieval systems returning hundreds of chunks. Do not use this prompt when you can implement application-layer pagination, streaming, or chunking that avoids truncation entirely. If the tool output is small enough to fit in context, prefer direct summarization without the overhead of truncation-aware instructions. Also avoid this prompt when the truncation point cuts through critical structured data like JSON objects or CSV rows, as the model may not be able to reconstruct partial records reliably.

Before deploying this prompt, instrument your tool-calling layer to detect truncation signals: HTTP 206 responses, truncated flags in API metadata, or context-window overflow events from your model gateway. The prompt works best when it receives explicit truncation metadata alongside the raw output. If your system silently truncates without signaling, the model cannot reliably distinguish between a complete result and a partial one. Pair this prompt with an eval harness that checks for critical information loss by comparing the summary against the full output in pre-production tests. In high-risk domains like healthcare or finance, always route truncated results through human review before the summary reaches end users.

This prompt template handles the common production scenario where a tool returns more data than the model's context window can accept. Instead of silently dropping information or hallucinating missing details, the prompt instructs the model to produce a prioritized summary that explicitly marks truncation boundaries and flags potentially critical omissions. The template is designed to be dropped into an existing tool-calling pipeline where the application layer has already detected that the raw tool output exceeds the available token budget and has performed initial truncation before passing the remainder to the model.

textCopy
You are processing a truncated tool output. The original result from [TOOL_NAME] exceeded the available context window and was cut to fit. Your job is to produce a useful summary without pretending you saw the full output.

## INPUT
[TRUNCATED_TOOL_OUTPUT]

## CONSTRAINTS
- Never fabricate data, values, or records that are not present in the truncated output.
- Do not imply completeness. If the output was cut mid-record, mid-list, or mid-field, state that explicitly.
- Prioritize information by [PRIORITY_CRITERIA] when deciding what to include in the summary.
- If the truncation point falls inside a critical section (as defined by [CRITICAL_SECTION_RULES]), flag it as high-severity.

## OUTPUT_SCHEMA
Return a JSON object with the following fields:
- "summary": string (the prioritized summary of what was retrieved)
- "truncation_point": string (description of where the output was cut, e.g., "after record 47 of an estimated 200", "mid-JSON object in the 'items' array")
- "missing_information": string[] (list of specific data categories, fields, or record ranges known to be missing)
- "critical_omission_risk": "high" | "medium" | "low" (likelihood that truncated content contains decision-critical information)
- "recommended_action": string (whether to re-query with narrower parameters, paginate, or proceed with available data)

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK_LEVEL
[RISK_LEVEL]

Adapt this template by replacing the square-bracket placeholders with your specific tool context. [PRIORITY_CRITERIA] should encode what matters most for your use case—for example, recency, severity, monetary value, or user-specified filters. [CRITICAL_SECTION_RULES] defines which parts of the output are too important to lose mid-stream, such as safety alerts, compliance fields, or primary key columns. [FEW_SHOT_EXAMPLES] should include at least one example where truncation occurred in a high-risk section and one where it occurred in a low-risk section, so the model learns to calibrate the critical_omission_risk field. Set [RISK_LEVEL] to "high" if the tool output contains regulated, financial, clinical, or safety-significant data, and ensure the downstream application routes high-risk summaries for human review before any action is taken.

Integrating the Tool Output Truncation Handling Prompt into a production system requires more than copying the template. The harness must detect truncation before the prompt runs, manage context-window limits, validate the summary output, and decide whether the result is safe to show to a user or feed into a downstream agent. This section covers the application-layer scaffolding that makes the prompt reliable under real load, including pre-processing checks, model selection, output validation, retry logic, and logging for auditability.

Pre-processing and truncation detection. Before the prompt is assembled, the application must determine whether the raw tool output actually exceeds the available context budget. Implement a token-counting step using the target model's tokenizer (e.g., tiktoken for OpenAI models or the Anthropic token-counting API). Compare the tool output token count against the remaining context window after accounting for the system prompt, conversation history, and the truncation-handling prompt template itself. If the output fits comfortably, skip the truncation prompt and pass the full result to the standard summarization path. If it exceeds the budget, inject the raw output into the [FULL_TOOL_OUTPUT] placeholder and set [AVAILABLE_TOKEN_BUDGET] to the remaining capacity. For multi-turn applications, recalculate the budget on each turn, as prior messages consume tokens. Model choice: Prefer models with large context windows (e.g., GPT-4 Turbo, Claude 3 Opus) for the truncation prompt itself, since the prompt must ingest a large output and produce a structured summary. If using a smaller model for cost reasons, apply an aggressive pre-truncation step that keeps only the first N characters and last M characters of the tool output before passing it to the prompt, and log a warning that mid-output data may be lost.

Output validation and schema enforcement. The prompt template requests a JSON output with summary, truncated_sections, preserved_key_facts, and confidence fields. The harness must validate that the model returns parseable JSON matching this schema. Use a strict JSON schema validator (e.g., jsonschema in Python, zod in TypeScript) to check required fields, types, and value ranges. If truncated is true but truncated_sections is empty, flag the response for review. If confidence is below a configurable threshold (e.g., 0.7), route the output to a human reviewer or a secondary model for verification. Retry and fallback logic: If the model returns malformed JSON, retry up to two times with an error message appended to the prompt (e.g., "Your previous response was not valid JSON. Return ONLY the JSON object."). If retries are exhausted, fall back to a simpler truncation strategy: return the first N characters of the tool output with a generic truncation notice, and log the failure for investigation. Never pass a raw, unsummarized tool output to the user if it exceeds the display or context budget.

Logging, observability, and audit. Every truncation event should produce a structured log record containing: the original tool output token count, the available budget, the model used, the generated summary, the confidence score, whether validation passed, and any retry attempts. This log is essential for debugging cases where critical information was lost. For high-risk domains (healthcare, finance, legal), require human review of any truncation summary where confidence < 0.9 or where preserved_key_facts is empty. Wire the truncation prompt into your existing observability stack so that truncation frequency, confidence distributions, and validation failure rates are visible on dashboards. What to avoid: Do not silently truncate tool outputs without logging. Do not assume the model's summary is complete—always cross-check that preserved_key_facts contains the fields your downstream workflow requires. Do not use the truncation prompt as a substitute for proper pagination or result filtering at the tool level; it is a safety net, not a primary data management strategy.