Idempotency Key

An idempotency key is a client-generated unique identifier, such as a UUID v4. The client (e.g., an AI agent) must create a new, random key for each distinct logical operation it intends to perform. This ensures the key is globally unique and not guessable. The server uses this key to deduplicate incoming requests.

• Key Property: The key's uniqueness is the client's responsibility.
• Common Format: UUIDs (e.g., 550e8400-e29b-41d4-a716-446655440000).
• Instrumentation Hook: The key should be attached as a span attribute (e.g., idempotency.key) in distributed traces for auditability.

The core guarantee is that multiple identical requests (same key, parameters, and endpoint) result in the same server-side effect and return the same response. The server achieves this by caching the first response against the key.

• First Request: Executes normally; response and outcome are cached.
• Subsequent Identical Requests: Return the cached response without re-executing the operation.
• Critical for Retries: This pattern safely handles network timeouts, agent retry policies, and exponential backoff without causing duplicate charges, orders, or database entries.

Idempotency keys are not stored indefinitely. Servers typically maintain the cached response for a finite period, often 12 to 24 hours. This prevents unbounded storage growth and aligns with business logic where a 'repeat' of an operation after a long delay might be intentional.

• Expiry: After the window expires, a request with the same key is treated as a new, first request.
• Observability Signal: A cache miss on an expired key should be logged as a span event to distinguish it from a true first request.
• Configuration: The validity window is a server-side configuration, not client-controlled.

An idempotency key is tightly bound to the exact request parameters, HTTP method, and URL path. Changing any parameter while reusing a key typically results in a 409 Conflict or 422 Unprocessable Entity error, as the server detects a mismatch.

• Scope Definition: Key + Method + Path + Request Body = Idempotent Unit.
• Error Handling: Agents must be instrumented to catch and handle 409 errors, which indicate a client logic error.
• Telemetry: These errors are critical Service Level Indicator (SLI) signals for agent correctness and should trigger alerts.

The standard mechanism for transmitting the key is via the HTTP header Idempotency-Key. This keeps the business logic payload clean and allows middleware (like API gateways or instrumentation libraries) to process it uniformly.

• Header Propagation: For multi-service calls, the key may be propagated downstream via headers as part of trace correlation.
• Alternative Patterns: Some APIs use a custom header (e.g., X-Idempotency-Key) or a field in the JSON request body, though the header approach is preferred.
• Security: The header value should be logged in observability platforms with the same sensitivity as other request identifiers.

Idempotency keys are a cornerstone of agent behavior auditing. They provide a deterministic link between an agent's intent (a task) and the resulting external action.

• Trace Correlation: The key should be added as a span attribute on the root span of an agent's execution context, linking all related tool call spans.
• Audit Trail: In logging systems, the key allows precise reconstruction of 'what happened' despite network retries.
• Cost Attribution: When combined with cost attribution tags, idempotency keys ensure duplicate retries are not mistakenly billed, providing accurate agent cost telemetry.

A Retry Policy is a set of rules governing the automatic re-attempt of failed tool or API calls. It defines the conditions for a retry (e.g., on a timeout or a 5xx HTTP status), the maximum number of attempts, and the delay strategy between attempts. Idempotency keys are a foundational requirement for safe retries, ensuring that repeated calls do not cause duplicate side effects like charging a credit card twice.

• Key Components: Retryable error conditions, max retry count, backoff strategy.
• Integration with Idempotency: The policy must include the same idempotency key on all retry attempts for the same logical operation.

Exponential Backoff is a specific retry strategy where the wait time between consecutive retry attempts increases exponentially (e.g., 1s, 2s, 4s, 8s). This pattern is used in conjunction with a retry policy and idempotency keys to improve system resilience.

• Purpose: Reduces load on a potentially failing or overloaded external service, increasing the chance it can recover.
• How it Works: Each retry waits for base_delay * (2 ^ attempt_number) before executing, often with added jitter to prevent client synchronization.
• Critical Pairing: The idempotency key remains constant across all backoff-scheduled retries, guaranteeing the operation's final state is consistent.

The Circuit Breaker Pattern is a resilience design pattern that prevents an application from performing operations that are likely to fail. For tool calls, it programmatically fails fast when a dependency is unhealthy, allowing it time to recover.

• Three States: Closed (normal operation), Open (failing fast, no requests sent), Half-Open (allowing a test request to check for recovery).
• Interaction with Idempotency: When the circuit is Open or Half-Open, calls are not made to the external service. Therefore, idempotency keys are not consumed. The client must decide whether to cache the key for a later retry or fail the operation entirely.
• Observability Link: Circuit breaker state transitions (e.g., trip events) are critical telemetry signals captured alongside tool call spans.

Distributed Tracing is a method of observing requests as they propagate through a distributed system. For an agent making tool calls, a trace provides the full end-to-end context, showing how the idempotent call fits into the larger workflow.

• Core Unit: The Span represents a single operation, such as the execution of a specific tool call with its idempotency key.
• Trace Context: A unique trace ID is propagated, often in HTTP headers, linking spans from the agent, through intermediaries, to the external API and back.
• Key Metadata: The idempotency key should be recorded as a Span Attribute on the relevant span, making it queryable during debugging to correlate logs and API-side records.

A Dead Letter Queue (DLQ) is a holding queue for messages or tool call requests that cannot be processed successfully after multiple attempts. It is a last-resort mechanism for handling persistent failures in asynchronous systems.

• Use Case: If a tool call with an idempotency key fails repeatedly due to a downstream bug or invalid data, it may be moved to a DLQ after exhausting the retry policy.
• Analysis & Replay: Engineers can inspect the failed request in the DLQ, including its idempotency key, diagnose the root cause, and potentially replay it once the issue is fixed.
• Idempotency Consideration: Replaying from a DLQ must preserve the original idempotency key to maintain the guarantee against duplicate processing.

A Service Level Objective (SLO) is a target value or range for a Service Level Indicator (SLI), forming a reliability contract. For tool calls, common SLOs are defined for success rate and latency.

• Example SLO: "99.9% of tool calls must complete successfully (HTTP 2xx) under 500ms P95 latency."
• Idempotency's Role: Idempotency keys directly support SLOs related to correctness and data integrity. They ensure that retries—which are essential for achieving high success rates in the face of transient errors—do not corrupt system state.
• Error Budget: The allowable unreliability (1 - SLO) guides how aggressively to use retries. Idempotency ensures retries consume latency/error budget without consuming data integrity budget.