Tool Call Chain of Thought Reasoning Prompt

This prompt is for engineering teams who need to make AI tool-calling decisions auditable, debuggable, and reviewable. Use it when a model must select from multiple available tools and you need a traceable record of why it chose a specific tool and how it populated the arguments. This is critical for regulated workflows, customer-facing features where incorrect tool calls have user-visible consequences, and any system where an operator needs to inspect the model's decision logic after the fact.

The ideal implementation embeds this prompt in a tool-selection harness where the model first outputs a structured reasoning block before emitting the actual tool call. Your application should parse the reasoning, log it alongside the tool call for audit trails, and route it to a human review queue when the confidence is low or the risk level is high. For example, in a financial operations system, a transfer_funds tool call should always produce a reasoning trace explaining why that specific transfer endpoint was chosen, how the amount was derived, and why alternative tools like schedule_payment were rejected. This trace becomes part of the compliance record.

Do not use this prompt when latency is the primary constraint and the tool call is low-risk, or when the model has only one available tool and no selection reasoning is required. Adding chain-of-thought reasoning to every tool call in a high-throughput, low-stakes pipeline—such as a content tagging system with a single apply_tag function—will waste tokens and increase response times without providing meaningful audit value. Reserve this pattern for decision points where the cost of a wrong tool call justifies the cost of explaining it.

This template is designed to be pasted into your system prompt or prefixed to a user task. It instructs the model to perform explicit chain-of-thought reasoning about which tool to call, why that tool is appropriate, and what arguments are required, all before the actual tool call is emitted. The primary value is auditability: when a tool call fails, produces an unexpected side effect, or is selected incorrectly, the reasoning block provides a clear trace for debugging the model's decision process. This is critical for high-stakes applications in finance, healthcare, or infrastructure where an incorrect tool call is not just a nuisance but a potential liability.

textCopy
You are an AI assistant with access to a set of tools. Your task is to decide which tool, if any, to call to satisfy the user's request.

Before you emit any tool call, you MUST produce a reasoning block inside <thinking> tags. This block must contain:
1. **User Intent**: Restate the user's core goal in one sentence.
2. **Tool Candidates**: List the tools that could potentially satisfy this request from the provided [TOOLS] list. If none apply, state that explicitly.
3. **Selection Rationale**: Explain why the chosen tool is the best fit and why others were rejected. Reference specific capabilities from the tool descriptions.
4. **Argument Plan**: List each required argument for the chosen tool, its expected type, and where you will source its value (from the user's message, from [CONTEXT], or a default). If a required argument is missing, note it here.
5. **Risk Check**: If the tool has a [RISK_LEVEL] of 'high' or 'destructive', confirm that the action is reversible or that user confirmation has been obtained.

After the </thinking> tag, emit the tool call in the exact format specified by the tool definition. If no tool is appropriate, respond to the user directly without a tool call.

[TOOLS]
[CONTEXT]
[EXAMPLES]
[CONSTRAINTS]
[RISK_LEVEL]

To adapt this template, replace the square-bracket placeholders at runtime. [TOOLS] should contain your serialized tool definitions (JSON for OpenAI, XML for Anthropic). [CONTEXT] is any retrieved evidence or user-specific data. [EXAMPLES] should include one or two few-shot demonstrations of correct reasoning blocks followed by correct tool calls, covering both straightforward and ambiguous cases. [CONSTRAINTS] can enforce rules like 'never call the DELETE tool without confirmation.' [RISK_LEVEL] should be a per-tool annotation or a global policy string. After assembly, validate the final prompt to ensure no placeholders remain unresolved and that the tool schemas are syntactically valid for your target model. For high-risk deployments, log the full <thinking> block alongside the tool call for post-incident review.

This prompt is not a standalone chat interaction; it is a reasoning gate that sits between the tool selection layer and the tool execution layer in your application. The harness must capture the model's reasoning text, parse the final tool call from the output, validate the call against the provided [TOOLS] schema, and only then proceed to execution. The reasoning trace should be logged as structured metadata alongside the tool call, not discarded, because it is the primary artifact for debugging incorrect tool selections and for satisfying audit requirements in regulated workflows.

Implement a wrapper function that accepts the assembled prompt and a list of valid tool definitions. After inference, use a strict parser to extract the tool call from the model's response. If the parser cannot find a valid function name or the arguments fail JSON schema validation against the target tool's parameters, do not execute the call. Instead, route the failure to a retry path that injects the validation error and the original reasoning trace back into the prompt, asking the model to correct its call. Set a hard retry limit (typically 2-3 attempts) before escalating to a human review queue or returning a structured error to the user. For high-risk tool calls where [RISK_LEVEL] is set to high, always require a synchronous human approval step after successful parsing but before execution, presenting the reasoning trace and the parsed arguments for review.

Model choice matters here. This prompt works best with models that have strong instruction-following and structured output capabilities. If you are using a model that does not natively separate reasoning from the tool call, you may need to post-process the output with a regex or delimiter-based split. Log every attempt—including the raw prompt, the full model response, the parsed tool call, validation results, and the final execution outcome—to an observability platform. This trace data is essential for running the eval checks described in the evaluation section, such as measuring reasoning-tool alignment and argument accuracy. Never deploy this prompt in a path where a hallucinated tool call could trigger an irreversible side effect without a human in the loop.