Tool Call Hallucination Recovery Prompt for Non-Existent Functions

Use this prompt inside a retry loop when your agent harness catches a 'function not found' error from the tool dispatcher. The model has generated a tool call with a function name that does not match any entry in your available tool catalog. Instead of failing silently, retrying the same hallucinated call, or returning a generic error to the user, this prompt instructs the model to perform semantic recovery. It compares the hallucinated function name and its intended payload against your actual tool catalog, identifies the closest match based on capability similarity, and adapts the arguments to fit the real tool's signature. The goal is to preserve the user's original intent while grounding the execution in tools that actually exist.

This prompt is designed for agent harnesses where tool selection is critical and hallucinations break execution pipelines. It assumes you have access to three pieces of context at the point of failure: the full tool catalog with names and descriptions, the original user intent or conversation history that led to the tool call, and the exact hallucinated call payload including the invented function name and its arguments. The prompt forces the model to output a structured decision: either a corrected tool call mapped to an existing tool, or an escalation request when no reasonable match exists. A configurable confidence threshold determines whether substitution is safe or whether the system should request human clarification. Do not use this prompt for simple schema validation errors, missing required parameters, or type mismatches on otherwise valid function names—those failures have their own dedicated recovery playbooks in this pillar.

Before wiring this into production, define your substitution confidence threshold. A low threshold risks mapping hallucinated calls to inappropriate tools and executing unintended actions. A high threshold increases escalation frequency, which may degrade user experience if overused. Test against a golden set of known hallucinations to calibrate the threshold. Also ensure your tool catalog descriptions are detailed enough for semantic matching—vague or overlapping descriptions will produce ambiguous mappings. After implementing this prompt, monitor the ratio of successful substitutions to escalations, and audit a sample of substituted calls to verify that the adapted arguments preserve user intent without introducing silent errors.

This prompt template is designed to be injected as a system or developer message on the retry turn immediately after the agent harness catches a 'tool not found' or equivalent error. Its job is to force the model to reconcile its attempted call against the actual tool catalog, producing a corrected call or a structured escalation. The template uses square-bracket placeholders that your harness must populate with runtime data: the hallucinated call, the error message, the full list of available tool definitions, and your substitution confidence threshold.

textCopy
You attempted to call a tool named [HALLUCINATED_TOOL_NAME] with arguments [HALLUCINATED_ARGUMENTS].
This tool does not exist. The system returned the following error:
[ERROR_MESSAGE]

Here is the complete list of available tools and their schemas:
[AVAILABLE_TOOLS_CATALOG]

Your task is to recover by doing ONE of the following:

1. SUBSTITUTE: If you can identify an available tool that can fulfill the original intent with high confidence (>= [CONFIDENCE_THRESHOLD]), output a corrected tool call using that tool's name and schema. Map the original arguments to the new tool's parameters as faithfully as possible. Do not invent new intent.

2. ESCALATE: If no available tool is a suitable substitute, or if your confidence in the mapping is below the threshold, output an escalation object that explains the gap and preserves the original intent for human review.

Output your decision in this exact JSON format:
{
  "decision": "substitute" | "escalate",
  "corrected_tool_call": {
    "tool_name": "string",
    "arguments": {}
  },
  "escalation": {
    "reason": "string",
    "original_intent": "string",
    "missing_capability": "string"
  },
  "confidence": 0.0
}

If you substitute, populate "corrected_tool_call" and leave "escalation" null. If you escalate, populate "escalation" and leave "corrected_tool_call" null. Always include the "confidence" field with a value between 0.0 and 1.0.

To adapt this template, replace the placeholders with runtime values from your agent harness. [HALLUCINATED_TOOL_NAME] and [HALLUCINATED_ARGUMENTS] come from the failed call payload. [ERROR_MESSAGE] is the raw error string from the tool execution layer. [AVAILABLE_TOOLS_CATALOG] should be a serialized JSON array of your full tool definitions, including names, descriptions, and parameter schemas—injecting only the relevant subset risks missing a valid substitute. Set [CONFIDENCE_THRESHOLD] to a float like 0.8; lower values increase substitution attempts but risk incorrect mappings, while higher values push more cases to escalation. After the model responds, validate the JSON structure and check that the decision field matches the populated object before executing the corrected call or routing the escalation. For high-risk domains, always require human review on escalation outputs before taking further action.

This prompt is not a standalone fix; it is a recovery step inside a tool-call execution loop. The harness must intercept a model response that contains a function name not present in the active tool catalog, extract the hallucinated name and arguments, and inject them into this prompt alongside the catalog of available tools. The prompt returns either a corrected tool call mapped to a real function or an escalation signal when no reasonable match exists. Do not call this prompt for schema validation failures or argument type errors—those have separate recovery playbooks. This prompt is specifically for the case where the model invented a tool that does not exist.

The harness should maintain a retry budget per turn, typically 1–2 recovery attempts for hallucination errors. On the first attempt, inject the full tool catalog with names, descriptions, and parameter schemas. If the recovery prompt returns a corrected call, validate that the new function name exists in the catalog and that the arguments match its schema before executing. If validation fails or the prompt returns an escalation signal, do not retry further—surface the failure to the user or a human reviewer with the original hallucinated call, the attempted correction, and the validation error. Log every recovery attempt with the hallucinated function name, the suggested substitution, and the confidence score for post-deployment analysis. This log becomes your primary signal for identifying gaps in your tool descriptions or model fine-tuning needs.

Model choice matters here. Smaller or older models are more prone to tool hallucination and may also struggle with the recovery prompt itself. Test this prompt against your specific model and tool catalog size. If your catalog exceeds 20–30 tools, consider injecting only the top-N relevant tools by embedding similarity to the user intent or the hallucinated function name, rather than the full catalog. This reduces prompt noise and improves substitution accuracy. Always include a confidence threshold in the harness: if the recovery prompt's suggested substitution confidence falls below your defined threshold (start with 0.7 and tune based on production data), treat it as an escalation rather than risking a second invalid call that wastes latency and compute.

Wire the escalation path carefully. When the recovery prompt escalates, the harness should format a structured message for the user or operator that includes the original user intent, the hallucinated function name, the attempted arguments, and a clear statement that no matching tool was found. In customer-facing agents, this might surface as a clarification question. In internal automation, it might route to a dead-letter queue for manual inspection. Never silently drop a hallucinated call—it hides model behavior that will degrade trust and makes debugging impossible. Pair this harness with an observability dashboard that tracks hallucination rate per tool catalog version so you can detect regressions when you add, remove, or rename tools.