Inferensys

Prompt

Tool Call Repair with Retry Budget Awareness Prompt Template

A practical prompt playbook for embedding retry budget context into tool call repair prompts, adjusting strategy from aggressive inference to conservative escalation as attempts run out.
Strategy workshop with sticky notes and AI roadmap diagrams on glass wall, collaborative planning session.
PROMPT PLAYBOOK

When to Use This Prompt

Defines the operational context for embedding retry budget awareness into a tool call repair step within a production agent loop.

This playbook is for production engineers who need to repair malformed or invalid tool calls within an agent loop without aborting the entire execution. The core job-to-be-done is recovering from a schema validation failure on a function call generated by an LLM, where the system has a finite number of retry attempts before it must escalate or fail gracefully. The ideal user is an agent platform developer or backend engineer who already has a defined tool schema, a validation layer that produces specific error messages, and a retry budget (e.g., 3 attempts total). The required context includes the original user request, the conversation history, the invalid tool call JSON, the exact validation error, and the full tool schema. You should not use this prompt for initial tool call generation, for systems without a defined retry budget, or as a substitute for fixing a fundamentally broken system prompt that causes persistent tool call errors.

The prompt template embeds the remaining retry budget directly into the repair instruction, causing the model to shift its strategy based on how many attempts are left. On an early retry (e.g., attempt 1 of 3), the model is instructed to infer aggressively from conversation context and fix the issue with minimal changes. On a final attempt (e.g., attempt 3 of 3), the model is instructed to escalate conservatively—either by requesting human intervention, returning a safe default, or producing a structured error that the agent loop can handle without further retries. This prevents the common failure mode where a repair loop burns through all retries making the same type of mistake. The prompt requires the following placeholders: [INVALID_TOOL_CALL] for the malformed JSON, [VALIDATION_ERROR] for the specific schema violation message, [TOOL_SCHEMA] for the expected function definition, [CONVERSATION_CONTEXT] for the preceding turns, [RETRY_ATTEMPT] for the current attempt number, and [MAX_RETRIES] for the total budget.

Before implementing this prompt, ensure your agent framework can accurately track retry counts and inject them into the repair step. A common failure mode is a mismatch between the actual retry count in the orchestrator and the count passed to the prompt, which causes the model to use the wrong repair strategy. You should also implement structured logging that captures the repair attempt number, the model's chosen strategy (inferred vs. escalated), and whether the repaired call passed validation. This data is critical for tuning the retry budget and identifying whether the root cause is a prompt issue, a schema ambiguity, or a model capability gap. After reading this section, proceed to the prompt template to copy and adapt the repair instruction, then review the implementation harness for wiring it into your agent loop with proper validation and observability.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Tool Call Repair with Retry Budget Awareness prompt template delivers value and where it introduces risk. Use these cards to decide if this pattern fits your agent architecture.

01

Good Fit: Multi-Step Agent Loops

Use when: your agent makes tool calls in a loop and a single failure shouldn't abort the entire task. Guardrail: wire the retry budget into the agent state so the repair prompt can adjust its strategy based on remaining attempts.

02

Bad Fit: Single-Shot Tool Calls

Avoid when: the tool call is a one-shot request with no retry loop. Guardrail: if there's no budget to track, use a simpler repair prompt without retry-count awareness to reduce prompt complexity and token overhead.

03

Required Input: Retry Budget State

Use when: you can pass [RETRY_COUNT], [MAX_RETRIES], and [REMAINING_ATTEMPTS] into the prompt. Guardrail: if the agent framework doesn't expose retry state, add a lightweight counter wrapper before adopting this prompt.

04

Operational Risk: Aggressive Inference on Early Retries

Risk: early retries may infer missing arguments too aggressively, introducing hallucinated values. Guardrail: require source grounding for inferred values on retries 1-2 and switch to conservative escalation on the final attempt.

05

Operational Risk: Budget Exhaustion Without Escalation

Risk: the repair loop burns all retries without producing a valid call. Guardrail: the final-attempt branch must produce a structured escalation payload instead of another repair attempt, so the agent can hand off to a human or fallback path.

06

Bad Fit: Latency-Sensitive Paths

Avoid when: the tool call is on a critical path with tight latency SLOs. Guardrail: if multiple repair rounds exceed your latency budget, consider a single repair attempt with immediate escalation or a faster fallback model.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A production-ready template for repairing malformed tool calls with explicit awareness of the remaining retry budget.

This template is designed to be injected into your agent's repair step when a tool call fails validation. Unlike a generic repair prompt, it explicitly incorporates the [RETRY_BUDGET]—the number of remaining attempts—to adjust the model's strategy. On early retries, the prompt encourages aggressive, context-based inference to fix the issue quickly. On the final attempt, it forces a conservative fallback to a safe, no-op, or clarification request, preventing an infinite loop of hallucinated repairs. The prompt expects the original failing arguments, the validation error, and the tool's schema to be provided as dynamic context.

text
You are an expert tool-call repair agent inside an autonomous system. Your goal is to fix a malformed function call so it passes schema validation.

# CONTEXT
- Remaining repair attempts: [RETRY_BUDGET]
- The original user request and conversation history are provided below.

# FAILED TOOL CALL
- Tool Name: [TOOL_NAME]
- Invalid Arguments: [INVALID_ARGUMENTS_JSON]
- Validation Error: [ERROR_MESSAGE]

# EXPECTED SCHEMA
```json
[TOOL_SCHEMA_JSON]

REPAIR STRATEGY BASED ON RETRY BUDGET

  • If [RETRY_BUDGET] > 1: You may infer missing or incorrect values from the conversation context, tool description, and parameter descriptions. Prioritize getting a valid call executed. Be aggressive but grounded.
  • If [RETRY_BUDGET] == 1: This is the final attempt. Do not guess. If you cannot fix the arguments with absolute certainty, you must output a safe fallback. The fallback is a call to [CLARIFICATION_TOOL_NAME] asking the user for the specific missing or invalid parameters.

CONSTRAINTS

  • Do not invent values that are not strongly implied by the context.
  • Do not add parameters that are not in the schema.
  • Ensure all required parameters are present.
  • Output ONLY the repaired JSON tool call, enclosed in a ```json code block. No other text.

CONVERSATION CONTEXT

[CONVERSATION_HISTORY]

Repair the tool call now.

To adapt this template, replace the square-bracket placeholders with live data from your agent loop. [RETRY_BUDGET] should be an integer decremented by your orchestration framework before each repair attempt. [INVALID_ARGUMENTS_JSON] is the exact string that failed validation, and [ERROR_MESSAGE] is the raw output from your schema validator. The [TOOL_SCHEMA_JSON] must be the complete JSON Schema definition for the tool's parameters. Crucially, you must define a [CLARIFICATION_TOOL_NAME] in your tool registry that the agent can use to ask the user for help when a repair is impossible. Before deploying, test this prompt with a golden dataset of known malformed calls to ensure the retry-budget logic triggers the correct behavior at each stage.

IMPLEMENTATION TABLE

Prompt Variables

Placeholders required by the Tool Call Repair with Retry Budget Awareness prompt. Wire these from your agent loop, tool registry, and validation harness before invoking the repair model.

PlaceholderPurposeExampleValidation Notes

[FAILED_TOOL_CALL_JSON]

The malformed tool call payload that failed validation

{"tool":"search","args":{"q":123}}

Must be valid JSON string; parse check before insertion; null not allowed

[TOOL_SCHEMA]

The expected function definition with parameter types and constraints

{"name":"search","parameters":{"type":"object","properties":{"q":{"type":"string"}},"required":["q"]}}

Must be valid JSON schema; validate against tool registry; null not allowed

[VALIDATION_ERROR_MESSAGE]

The exact error returned by the tool call validator

TypeError: Expected string for parameter 'q', got number

Must be non-empty string; truncate at 500 chars; null not allowed

[REMAINING_RETRIES]

Number of repair attempts remaining in the current retry budget

2

Must be integer >= 0; parse as int; if null default to 0

[MAX_RETRIES]

Total retry budget allocated for this tool call

3

Must be integer > 0; parse as int; if null default to 1

[CONVERSATION_CONTEXT]

Recent conversation turns preceding the failed tool call

[{"role":"user","content":"Find docs about retry budgets"}]

Must be valid JSON array; null allowed if no context available

[REPAIR_STRATEGY_HINT]

Override hint for repair aggressiveness based on retry position

aggressive_inference

Must be one of: aggressive_inference, conservative_escalation, last_attempt; null allowed to use default strategy

[PREVIOUS_REPAIR_ATTEMPTS]

Array of prior repair attempts and their outcomes for this call

[{"attempt":1,"repair":"...","outcome":"still_invalid"}]

Must be valid JSON array; null allowed on first attempt; validate attempt count matches budget consumed

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the retry-budget-aware repair prompt into an agent loop with validation, logging, and escalation.

The retry-budget-aware repair prompt is designed to sit inside a structured agent loop, not as a one-off correction call. The harness must track the remaining retry budget, pass it into the prompt as [REMAINING_ATTEMPTS], and enforce the escalation path when the budget reaches zero. This means the calling code should maintain a counter decremented on each repair attempt, inject the current count into the prompt template, and route the final attempt's output to a human review queue or a dead-letter log rather than looping indefinitely.

Validation is the gate before and after repair. Before invoking the repair prompt, validate the original tool call against the expected schema using a JSON Schema validator or the tool platform's native validation. Capture the exact validation error message and the failing argument payload—these become [VALIDATION_ERROR] and [FAILING_TOOL_CALL] in the prompt. After the repair prompt returns a corrected call, re-validate it against the same schema. If validation passes, execute the tool call and log the repair attempt count, latency, and whether it was an early-retry aggressive fix or a final-attempt conservative escalation. If validation fails again and budget remains, loop back with the new error. If budget is exhausted, route to the escalation path defined in [ESCALATION_ACTION].

Model choice matters for repair quality. Early retries (high remaining budget) benefit from faster, cheaper models that can attempt aggressive inference—filling missing fields, normalizing enums, and fixing syntax. Final retries (budget at 1 or 0) should use a more capable model with stronger instruction-following, since the prompt instructs conservative escalation: returning a safe fallback, requesting clarification, or aborting with a structured error. Implement model routing in the harness by checking [REMAINING_ATTEMPTS] before selecting the model endpoint. Log which model handled each repair attempt so you can measure whether the routing strategy improves repair success rates.

Observability is non-negotiable. For every repair attempt, emit a structured log event containing: the original tool call, the validation error, the remaining budget, the repair prompt version, the model used, the corrected output, the re-validation result, and the final disposition (executed, escalated, or abandoned). This log becomes your repair quality dataset. Use it to track metrics like repair success rate by attempt number, hallucination rate in repaired arguments, and escalation frequency. Set alerts on sudden increases in final-attempt escalations—this often signals a model behavior change or a schema update that the repair prompt hasn't been adapted to handle.

Avoid wiring this prompt into a blind retry loop without budget awareness. The most common production failure mode is an agent that retries indefinitely, burning tokens and latency on unfixable tool calls. The [REMAINING_ATTEMPTS] injection is what prevents this. Test your harness by feeding it intentionally unfixable tool calls—missing required arguments with no context to infer from, completely hallucinated tool names, or schema violations that require information the agent doesn't have. Confirm that when budget hits zero, the system escalates rather than retrying. Pair this with a dead-letter queue or review interface so escalated cases don't silently disappear.

IMPLEMENTATION TABLE

Expected Output Contract

Fields, format, and validation rules for the repair response. Use this contract to validate the model's output before applying the repaired tool call.

Field or ElementType or FormatRequiredValidation Rule

repaired_tool_call

object

Must be valid JSON parseable. Must contain 'name' (string) and 'arguments' (object) keys matching the target tool schema.

repair_strategy

string (enum)

Must be one of: 'aggressive_inference', 'conservative_repair', 'minimal_fix', 'escalate'. Strategy must align with [RETRY_COUNT] and [MAX_RETRIES].

changes_summary

array of objects

Each object must have 'field' (string path), 'original_value' (any), 'repaired_value' (any), and 'reason' (string). Array must not be empty if any argument was modified.

confidence_score

number (0.0-1.0)

Must be a float between 0.0 and 1.0 inclusive. Score below 0.7 on final retry must trigger escalation per [ESCALATION_THRESHOLD].

validation_errors_resolved

array of strings

Must list each original validation error message that was addressed. If empty, original errors were not resolved; escalate.

remaining_retries

integer

Must equal [RETRY_COUNT] minus 1 after this repair. If negative, halt retry loop and escalate.

escalation_required

boolean

Must be true if confidence_score < [ESCALATION_THRESHOLD] or remaining_retries <= 0. False otherwise.

escalation_reason

string or null

Required when escalation_required is true. Must cite specific reason: low confidence, exhausted retries, or unresolvable schema conflict.

PRACTICAL GUARDRAILS

Common Failure Modes

What breaks first when embedding retry budget awareness into tool call repair prompts, and how to guard against it.

01

Retry Budget Leakage

What to watch: The model ignores the [REMAINING_RETRIES] context and applies the same aggressive repair strategy on every attempt, burning through the budget without improving outcomes. Guardrail: Add a hard constraint in the prompt: 'If [REMAINING_RETRIES] <= 1, you MUST return a safe fallback or escalate. Do not attempt a speculative repair.' Validate this behavior in evals by mocking a final retry state and asserting the output is a structured escalation, not a guessed fix.

02

Aggressive Inference on Final Attempt

What to watch: On the last retry, the model hallucinates missing required arguments or invents plausible values to force a successful tool call, leading to silent data corruption. Guardrail: Modify the system instructions for the final retry tier to explicitly forbid filling required fields with inferred data. The prompt must instruct: 'If a required field is missing and cannot be sourced from the conversation history, output a structured error, not a guess.'

03

Context Window Exhaustion

What to watch: The repair prompt includes the full conversation history, the original tool call, the validation error, and the retry budget context. On deep agent loops, this can exceed the model's context window, causing truncation of the retry instructions themselves. Guardrail: Implement a context-packing strategy. Truncate the conversation history to the last N turns relevant to the tool call. Always place the [RETRY_BUDGET_CONTEXT] and [REPAIR_INSTRUCTIONS] at the very end of the prompt to prevent them from being truncated first.

04

Repair-Only Stagnation

What to watch: The repair prompt fixes the syntax but not the semantic intent. The model corrects a JSON error, but the underlying tool choice or parameter value remains wrong for the user's goal, causing the agent to loop uselessly. Guardrail: Include a pre-repair check step. On retry 2+, prompt the model to first evaluate: 'Is this the correct tool for the user's goal? If not, output a tool-change suggestion instead of a parameter repair.' This prevents fixing the wrong thing perfectly.

05

Budget-Aware Prompt Drift

What to watch: The dynamic insertion of [RETRY_BUDGET_CONTEXT] changes the prompt's structure enough that the model's output format shifts unexpectedly (e.g., it starts wrapping the repaired JSON in markdown or adding explanatory text). Guardrail: Use a strict output schema validator immediately after the repair prompt. The schema should reject any output that isn't a pure JSON object matching the tool's parameters. If validation fails, increment the retry counter and feed the schema error back into the next repair attempt.

06

Escalation Ambiguity

What to watch: When the budget is exhausted, the model outputs a vague natural language message like 'I couldn't fix the error' instead of a machine-readable escalation signal, breaking the agent's control loop. Guardrail: Define a strict escalation contract. The prompt must specify: 'When escalating, output ONLY a JSON object with {"action": "escalate", "reason": "..."}'. The agent harness must parse this and route to a human or a fallback workflow without attempting further retries.

IMPLEMENTATION TABLE

Evaluation Rubric

Criteria for evaluating the quality of a retry-budget-aware tool call repair before shipping to production. Each criterion maps to a pass standard, a failure signal, and a concrete test method.

CriterionPass StandardFailure SignalTest Method

Schema Compliance

Repaired arguments pass strict validation against the target tool's JSON schema with zero errors.

Validation error on a required field, type mismatch, or extra hallucinated parameter.

Automated schema validator run on the repaired output; diff against original failing call to confirm only invalid fields changed.

Retry Budget Awareness

Repair strategy shifts from aggressive inference to conservative escalation as [REMAINING_RETRIES] decreases. On final attempt, no hallucinated values are injected.

Hallucinated default values appear on the final retry; repair prompt ignores the retry budget context.

Parameterize test with [REMAINING_RETRIES]=1 and [REMAINING_RETRIES]=3; assert that final-attempt repairs contain null or explicit abstention markers for unrecoverable fields.

Context Grounding

Inferred values for missing arguments are traceable to [CONVERSATION_HISTORY], [TOOL_DESCRIPTION], or [ERROR_MESSAGE]. No invented entities.

Repair inserts a plausible but unsupported value not present in any provided context.

Human review of 20 repair samples; automated check that repaired values appear verbatim in the input context or are marked as [DEFAULT].

Error Message Utilization

Repair directly addresses the specific error in [ERROR_MESSAGE]. If error says 'missing required field X', repair adds field X.

Repair fixes a different field than the one flagged in the error, or repeats the same invalid value.

Parse [ERROR_MESSAGE] for field name; assert repaired output contains a new value for that exact field.

Preservation of Valid Arguments

All arguments that passed initial validation remain unchanged in the repaired call.

A valid argument is altered, removed, or corrupted during the repair of an unrelated field.

Diff valid arguments before and after repair; assert zero differences in the valid subset.

Escalation Signal on Unrecoverable Errors

When [REMAINING_RETRIES]=0 and repair is impossible, output includes an explicit escalation flag and a human-readable reason.

Model fabricates a plausible but invalid repair on the final attempt instead of escalating.

Assert that output contains [ESCALATION_REQUIRED]=true and a non-empty [ESCALATION_REASON] when schema validation still fails after repair.

Repair Latency Budget

Repair prompt completes within the configured [REPAIR_TIMEOUT_MS] threshold.

Repair exceeds the timeout, causing agent loop delay or abort.

Instrument repair call with a timer; assert p95 latency across 100 calls is below the threshold.

Idempotency

Applying the same repair prompt to the same failing input twice produces identical repaired arguments.

Two repair attempts on the same input yield different field values or structures.

Run repair twice on 10 fixed failure cases; assert byte-level equality of the repaired JSON.

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Implement a two-phase repair strategy based on [RETRY_BUDGET_REMAINING]. For early retries (budget > 1), use an aggressive repair prompt that attempts full correction with schema context. For the final retry (budget == 1), switch to a conservative prompt that strips non-essential arguments and applies safe defaults for missing required fields. Add structured logging: repair_attempt, strategy_used, budget_remaining, repair_success.

code
IF [RETRY_BUDGET_REMAINING] > 1:
  Use aggressive repair with full [TOOL_SCHEMA]
ELSE:
  Use conservative repair: strip optional fields, apply [SAFE_DEFAULTS]

Watch for

  • Silent format drift across retries where the output shape changes subtly
  • Conservative mode dropping required fields that have no safe default
  • Retry budget exhaustion without escalation to human review
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.