Circuit Breaker Pattern

A Retry Policy is a set of rules governing the automatic re-attempt of failed operations, such as tool or API calls. It defines the conditions for a retry (e.g., on transient errors like timeouts or HTTP 5xx statuses), the maximum number of attempts, and the delay strategy between attempts. A well-designed retry policy is a prerequisite for a circuit breaker, as the breaker monitors these retry failures to determine when to open.

• Key Parameters: Max retries, retryable status codes/errors.
• Common Strategy: Often paired with Exponential Backoff to prevent overwhelming a recovering service.

Exponential Backoff is a retry delay strategy where the wait time between consecutive retry attempts increases exponentially (e.g., 1s, 2s, 4s, 8s). This is a critical companion to a retry policy and circuit breaker. It reduces load on a failing or recovering service, giving it time to stabilize, and increases the probability that a subsequent retry will succeed. The circuit breaker's 'half-open' state often uses a similar backoff logic for its probe requests.

• Formula: Delay = base_delay * (2 ^ attempt_number).
• Jitter: Randomization is often added to prevent synchronized retry storms from multiple clients.

The Bulkhead Pattern isolates elements of an application into pools so that if one fails, the others continue to function. Inspired by ship compartments, it prevents a failure in one dependency from cascading and consuming all system resources (like threads or connections). While a circuit breaker fails fast on a specific failing dependency, a bulkhead contains the failure's resource impact.

• Implementation: Using separate thread pools, connection pools, or even processes for different external services.
• Synergy: Used alongside circuit breakers to provide layered fault isolation.

A Fallback Strategy defines an alternative course of action when a primary operation fails and a circuit breaker is open. It is the 'graceful degradation' mechanism that maintains some system functionality. For an agent's tool call, this could mean:

• Returning cached or stale data.
• Calling a secondary, less-capable API.
• Providing a default or placeholder response.
• Queuing the request for later processing.

The circuit breaker's open state triggers the fallback, ensuring the user or agent receives a timely, even if degraded, response instead of an error.

A Dead Letter Queue (DLQ) is a persistent holding area for messages or requests that cannot be processed after repeated failures. In the context of tool calls, when a circuit breaker is open and a fallback is not appropriate, the request can be placed in a DLQ. This allows for:

• Manual Inspection: Engineers can analyze the failed payloads.
• Delayed Retry: Requests can be replayed later when the dependency is healthy.
• Audit Trail: Maintaining a record of critical failures for compliance. It acts as a safety net for operations that must not be silently dropped.

Idempotency is the property of an operation whereby performing it multiple times yields the same result as performing it once. This is a critical design consideration when using retries and circuit breakers. If a tool call (e.g., chargeCreditCard) is retried after a timeout, it must not cause duplicate charges. Implementing idempotency often involves:

• Idempotency Keys: Unique client-generated keys sent with the request that the server uses to deduplicate.
• Designing Safe Operations: Using HTTP PUT for updates, or designing APIs where repeated execution has no additional side effects. Ensuring idempotency makes retry policies and circuit breaker recovery probes safe to execute.