Idempotency Key

The primary guarantee of an idempotency key is exactly-once execution. When a network timeout occurs, the client retries the request with the same key. The server recognizes the key, returns the stored result of the original operation, and discards the duplicate request without re-executing the business logic. This transforms an inherently unreliable network into a reliable transactional boundary.

The client bears full responsibility for generating a globally unique key before the first attempt. Common implementations use:

• UUID v4: Random, 128-bit identifiers providing sufficient entropy.
• Content-based hash: A deterministic SHA-256 hash of the request payload, useful when the same input must always produce the same key. The key must be unique per operation, not per request, and is typically stored in the Idempotency-Key HTTP header.

The server manages a defined lifecycle for each key:

1. Receive: Extract the key from the request header.
2. Lock: Acquire an atomic lock on the key to prevent concurrent processing.
3. Lookup: Check a persistent store for an existing result.
4. Execute or Return: If new, process the request and store the response. If seen, return the cached response immediately.
5. Expire: Keys and their stored responses are purged after a configurable TTL, typically 24 hours.

Idempotency keys are distinct from optimistic concurrency mechanisms like ETags or version vectors. An idempotency key ensures a single operation is not duplicated, while an ETag ensures an operation fails if the underlying resource has changed. They solve different problems:

• Idempotency Key: 'Don't process this twice.'
• ETag/If-Match: 'Don't process this if the state has changed.' Both are often used together in robust APIs.

Common pitfalls in idempotency key implementations include:

• Key collision: Two different operations accidentally generate the same key. Mitigated by using UUID v4 with sufficient entropy.
• Partial execution: The operation succeeds but the response fails to persist. The retry must return the same outcome, requiring atomic write of result and response.
• Non-idempotent side effects: The key only covers the API boundary. If the downstream operation triggers an external webhook, that webhook may fire twice unless it also implements idempotency.

A message delivery guarantee ensuring that each operation is applied exactly one time, with no duplicates and no data loss. In distributed systems, this is the strongest consistency guarantee. Idempotency keys are the primary client-side mechanism to achieve this when the underlying transport (e.g., HTTP) provides at-most-once delivery. The system stores the operation result alongside the key, so retries return the cached outcome rather than re-executing the logic.

A concurrency control method that assumes conflicts are rare and validates data integrity only at commit time, rather than using locks. When paired with idempotency keys, the key acts as a unique constraint in the database. If two requests with the same key arrive concurrently, one succeeds and the other detects a conflict, returning the original result. This avoids the performance overhead of pessimistic locking while still preventing duplicate side effects.

A queue that stores messages that cannot be processed successfully after all retry attempts are exhausted. When an idempotent operation fails due to a non-transient error (e.g., invalid payload), the message is routed to the DLQ for manual inspection. The idempotency key is preserved in the DLQ metadata, allowing operators to safely replay the message once the root cause is fixed, without risking duplicate processing.

A resilience pattern that prevents cascading failures by detecting when a downstream service is unhealthy and failing fast rather than retrying. When combined with idempotency keys, the circuit breaker's state transition (closed → open → half-open) must be carefully coordinated. A retry with the same idempotency key after the circuit closes should still return the original result if the first attempt succeeded before the breaker tripped.

An architectural pattern where state changes are persisted as an immutable sequence of events. Idempotency keys integrate naturally here: the key is stored as part of the event metadata, and the event store uses it to deduplicate commands. If a command with a known key arrives, the system simply acknowledges it without appending a duplicate event to the log, preserving the integrity of the event stream.

A caching strategy where writes are applied to the cache first and asynchronously persisted to the backing store. Idempotency keys are critical here to prevent write amplification. If a cache flush fails and is retried, the idempotency key ensures the database applies the write exactly once. Without this, a retried flush could create duplicate records or double-increment counters in the persistent store.