Tool Use with Idempotency and Safe Retry Prompt Template

This prompt is for agent developers and platform engineers who are building AI systems that perform non-idempotent write operations—such as creating database records, sending emails, initiating payments, or updating external system state—where a duplicate call could cause double-spend, duplicate records, or corrupted data. The job-to-be-done is teaching the model, through concrete few-shot examples, to generate and attach idempotency keys, recognize already-completed operations, and decide when a retry is safe versus when it requires human intervention. The ideal user is someone integrating an LLM agent with production APIs and databases who already understands the basics of function calling but needs the model to handle the failure modes that turn a prototype into a reliable system.

Use this prompt when your agent's tool set includes at least one write operation that is not safe to repeat, when you have a defined idempotency key strategy (such as UUID generation or deterministic key derivation from input parameters), and when you can provide examples of both successful idempotent calls and failure scenarios. The prompt is designed for few-shot demonstration: you supply worked examples showing the model generating an idempotency_key before a write, detecting a 409 Conflict or duplicate response, and either returning the existing result or retrying with a corrected payload. Do not use this prompt for read-only agents, for systems where every operation is naturally idempotent, or when your tool definitions already enforce idempotency at the API gateway level and the model does not need to reason about it.

Before adopting this prompt, confirm that your tool schemas expose the idempotency_key parameter explicitly and that your API responses include status codes or fields the model can use to detect duplicates. If your tools do not return clear signals for 'already completed,' the model will hallucinate completion status. Pair this prompt with a validation harness that checks every tool call for the presence of an idempotency key on write operations, logs duplicate detection events, and escalates to a human reviewer when the model retries a write without a key or after receiving an ambiguous response. Start with a small set of 3–5 contrasting examples covering: a clean first attempt, a safe retry with the same key, an unsafe retry that should be blocked, and a duplicate detection that returns the cached result.

This template provides a system prompt and a set of few-shot demonstration slots designed to teach a language model the critical distinction between safe and unsafe retries for tool calls. Instead of relying on abstract rules, the prompt uses contrasting examples of correct idempotency key generation, conditional retry logic, and detection of already-completed operations. The primary goal is to prevent duplicate mutations—such as creating a second identical resource, charging a customer twice, or sending a duplicate notification—when an agent encounters a network error or timeout after a write operation.

markdownCopy
[SYSTEM_INSTRUCTION]
You are an agent that performs actions using external tools. Your primary directive is safety for write operations. Before calling any tool that creates, updates, or deletes data, you must generate a unique idempotency key. If a tool call fails with a retryable error (e.g., network timeout, 5xx status), you must check the tool's description to see if it supports idempotency keys. If it does, retry the call with the same idempotency key. If it does not, or if the error is a definitive failure (e.g., 409 Conflict, 422 Unprocessable Entity), you must not retry and instead report the failure to the user. Your response must be a JSON object with the keys "thinking", "tool_calls", and "final_response".

[TOOLS]
```json
[AVAILABLE_TOOL_SCHEMAS]

[EXAMPLES]

Example 1: Safe Retry with Idempotency

User: Create a new project named "Q3 Audit". Assistant Thought: The create_project tool is a write operation. I must generate an idempotency key. Assistant:

jsonCopy
{
  "thinking": "Generating an idempotency key for the project creation to ensure safety on retry.",
  "tool_calls": [
    {
      "id": "call_1",
      "name": "create_project",
      "arguments": {
        "name": "Q3 Audit",
        "idempotency_key": "proj-q3-audit-2024-001"
      }
    }
  ]
}

Tool Result (Error): {"error": "Network timeout", "status": 504} Assistant Thought: The error is a retryable server error. The create_project tool supports idempotency keys. I will safely retry with the same key. Assistant:

jsonCopy
{
  "thinking": "Retrying the project creation with the same idempotency key 'proj-q3-audit-2024-001' to avoid creating a duplicate project.",
  "tool_calls": [
    {
      "id": "call_2",
      "name": "create_project",
      "arguments": {
        "name": "Q3 Audit",
        "idempotency_key": "proj-q3-audit-2024-001"
      }
    }
  ]
}

Tool Result (Success): {"id": "123", "name": "Q3 Audit", "status": "active"} Assistant:

jsonCopy
{
  "thinking": "Project created successfully after a safe retry.",
  "final_response": "I've created the 'Q3 Audit' project for you. Its ID is 123."
}

Example 2: Unsafe Retry Prevention

User: Send the monthly invoice to client Acme Corp. Assistant Thought: The send_invoice tool is a write operation that triggers a financial action. I must generate an idempotency key. Assistant:

jsonCopy
{
  "thinking": "Generating an idempotency key for the invoice sending operation.",
  "tool_calls": [
    {
      "id": "call_3",
      "name": "send_invoice",
      "arguments": {
        "client": "Acme Corp",
        "period": "2024-06",
        "idempotency_key": "inv-acme-2024-06-001"
      }
    }
  ]
}

Tool Result (Error): {"error": "Invoice for period 2024-06 already sent.", "status": 409} Assistant Thought: The error is a 409 Conflict, indicating the operation already succeeded. This is not a retryable error. I must not retry and should inform the user. Assistant:

jsonCopy
{
  "thinking": "The 409 Conflict error indicates the invoice was already sent. Retrying would be unsafe and could result in a duplicate charge. I will report this to the user.",
  "final_response": "It looks like the June 2024 invoice for Acme Corp has already been sent. I did not resend it to avoid a duplicate charge. Can you verify on your end?"
}

Example 3: Non-Idempotent Tool Failure

User: Log a debug event "cache-cleared". Assistant Thought: The log_event tool is a write operation but its schema does not list an idempotency_key parameter. It is not idempotent. Assistant:

jsonCopy
{
  "thinking": "The log_event tool does not support idempotency. I will attempt the call once.",
  "tool_calls": [
    {
      "id": "call_4",
      "name": "log_event",
      "arguments": {
        "event_type": "debug",
        "message": "cache-cleared"
      }
    }
  ]
}

Tool Result (Error): {"error": "Internal server error", "status": 500} Assistant Thought: The tool failed with a server error, but it does not support idempotency keys. Retrying could cause a duplicate log entry, which is low-risk, but the principle is to not retry non-idempotent writes. I will report the failure. Assistant:

jsonCopy
{
  "thinking": "The log_event tool failed and does not support safe retries. I will report the failure to the user.",
  "final_response": "I wasn't able to log the 'cache-cleared' event due to a server error. The operation was not retried because the logging tool doesn't support idempotency keys."
}

[INPUT] [USER_QUERY]

To adapt this template for your own agent, replace the [AVAILABLE_TOOL_SCHEMAS] placeholder with the actual JSON Schema definitions for your tools. Crucially, ensure your tool definitions explicitly declare whether they support an idempotency_key parameter. The examples use a mix of idempotent and non-idempotent tools to teach the model the conditional logic. You should curate your own set of 3-5 examples that mirror your specific tool set and the most dangerous failure modes you want to prevent, such as double payments, duplicate resource creation, or redundant state transitions. The [USER_QUERY] placeholder is where the live user input will be injected at runtime, typically as the final message in the prompt array.

Before deploying, validate this prompt against a golden test set of user queries that should trigger retries, prevent retries, and escalate for human review. Pay special attention to edge cases where the model might hallucinate an idempotency key for a tool that doesn't support one, or fail to generate a key for a dangerous write operation. The eval harness should check the final JSON output for the presence of idempotency_key in the arguments of write operations and assert that retries only occur for retryable errors on idempotent tools.

This prompt is designed to sit inside an agent's tool-calling loop, directly before a write operation is executed. It is not a standalone chat prompt; it is a decision gate that receives the proposed tool call, the conversation history, and the idempotency registry state. The application must supply the [TOOL_CALL] object, the [IDEMPOTENCY_REGISTRY] of previously completed operations, and the [RETRY_POLICY] that defines which error classes are safe to retry. The model's job is to output a structured decision: execute, retry_with_key, skip_completed, or escalate_to_human. The application layer then enforces that decision, never the model alone.

Wire this prompt into your agent framework as a pre-execution hook. Before the agent calls any POST, PUT, PATCH, or DELETE operation, serialize the proposed tool call to JSON and pass it alongside the idempotency registry. The registry should be a simple key-value store (e.g., Redis or DynamoDB) mapping idempotency_key -> {status, result, timestamp}. The application must check the registry before calling the model: if a key exists with status completed, skip the model call entirely and return the cached result. If the key is missing or status is failed, invoke this prompt to decide the next action. After the model returns a decision, the application must validate the output against the [OUTPUT_SCHEMA]—reject any decision that invents a key not derived from the input or that classifies a non-retryable error as retryable. Log every decision, the model's rationale, and the final action taken to an audit trail.

For high-risk operations such as payment processing, user deletion, or permission changes, add a human review gate after the model's decision. If the model returns execute or retry_with_key for an operation above a configured risk threshold, queue the proposed action for human approval before the application executes it. The prompt's [RISK_LEVEL] parameter should be set by the application based on the tool's risk classification, not by the model. Never allow the model to downgrade the risk level. For retry logic, implement a circuit breaker: if the same idempotency key fails more than three times, force escalation regardless of the model's output. This prevents infinite retry loops that the model might incorrectly authorize. Finally, instrument the entire path with metrics: decision latency, escalation rate, retry rate, and human approval rate. These metrics will tell you when the prompt needs tuning or when the tool's error behavior has changed.