Circuit Breaker Pattern

The core logic that tracks consecutive or time-windowed failures against a configurable limit. When an agent's call to an external API or internal function fails repeatedly, the breaker increments an internal counter. Thresholds are typically set based on the criticality of the operation.

• Count-based: Trips after N consecutive failures (e.g., 5 failed tool calls).
• Time-based: Trips if failure rate exceeds X% in Y seconds.
• Hybrid: Combines both for nuanced detection of flapping services.

A circuit breaker operates as a finite state machine with three well-defined states that govern agent behavior:

• Closed: Normal operation. All requests pass through. Failures are counted.
• Open: The breaker has tripped. All requests are immediately rejected without execution, often returning a cached fallback or a predefined error. This prevents resource exhaustion.
• Half-Open: A testing state after a timeout. A limited number of probe requests are allowed to check if the underlying issue has resolved. Success resets to Closed; failure returns to Open.

The cooldown period defines how long the breaker stays in the Open state before transitioning to Half-Open. This gives the failing dependency time to recover from load or transient faults.

• Fixed Timeout: A static duration (e.g., 30 seconds) before probing.
• Exponential Backoff: The timeout doubles after each successive trip, preventing aggressive retries that could overwhelm a recovering service.
• Jitter: Randomization added to the timeout to avoid synchronized retry storms across multiple agents.

When the circuit is Open, the agent must not simply crash. It must execute a fallback strategy to maintain limited functionality. This is a direct implementation of graceful degradation.

• Cached Response: Return the last known good result from a local cache.
• Default Value: Provide a safe, static default that allows the workflow to continue non-critically.
• Alternative Service: Call a redundant or less-precise backup endpoint.
• Queued for Retry: Persist the request for later processing when the circuit closes.

Circuit breaker state transitions are high-signal events for agentic observability platforms. Every trip, reset, and fallback invocation must emit structured telemetry.

• Metrics: Track trip count, failure rate, and time spent in Open state.
• Traces: Correlate breaker events with specific agent task IDs to debug cascading failures.
• Alerts: Trigger immediate notifications to SRE teams when a critical circuit enters the Open state, potentially indicating a systemic outage.

Unlike naive retry logic, the circuit breaker prevents resource exhaustion. A simple retry loop consumes client CPU and memory while hammering a failing backend. The circuit breaker fails fast, preserving local resources and giving the downstream service a chance to recover.

• Retries: Optimistic, assume transient failure.
• Circuit Breaker: Pessimistic, assumes systemic failure after a threshold.
• Synergy: Retries are used inside the Closed state; the breaker stops them entirely when Open.

A companion pattern that works alongside circuit breakers to handle transient failures. When an operation fails, the system retries with progressively longer delays between attempts.

• Base delay: Starts at 100ms, doubles each retry
• Jitter: Adds random variance to prevent thundering herd problems
• Max retries: Typically capped at 3-5 attempts before circuit opens
• Idempotency requirement: Retried operations must be safe to execute multiple times

Without backoff, retries can amplify load on an already degraded service, turning a partial outage into a complete cascade failure.

Partitions system resources into isolated pools so that a failure in one component cannot consume all available threads, connections, or memory.

• Thread pool isolation: Dedicated thread pools per downstream dependency
• Connection pool partitioning: Separate connection limits per service
• Resource caps: Memory and CPU quotas enforced at the container or process level

In agentic systems, bulkheads prevent a single runaway agent from starving other agents of compute resources. Combined with circuit breakers, they provide defense in depth against cascading failures.

Controls the frequency at which an agent or service can make requests, preventing overload of downstream systems before failures occur.

• Token bucket algorithm: Allows bursts while enforcing long-term rate
• Sliding window: Tracks request counts over rolling time intervals
• Distributed enforcement: Coordinates limits across multiple agent instances

Rate limiting is a proactive sibling to the circuit breaker's reactive protection. While a circuit breaker trips after failures accumulate, rate limiting prevents the overload condition from arising in the first place.

A dedicated API endpoint that reports the current operational status of a service, enabling circuit breakers and load balancers to make informed routing decisions.

• Liveness check: Confirms the process is running and not deadlocked
• Readiness check: Indicates whether the service can accept requests
• Deep health probe: Validates connectivity to all downstream dependencies

Circuit breakers often use health checks to determine when to transition from half-open to closed state, testing if the downstream service has recovered before allowing full traffic resumption.

Defines alternative behavior when a circuit breaker is open and the primary operation cannot be executed. Fallbacks maintain graceful degradation rather than returning raw errors.

• Cached response: Serve stale but acceptable data from a local cache
• Default value: Return a safe, pre-defined result
• Alternative service: Route to a redundant or degraded backup endpoint
• Queued for later: Persist the request for deferred processing

In agentic systems, fallbacks prevent task failure propagation. An agent encountering an open circuit can continue its workflow with degraded but functional data rather than aborting entirely.

Sets maximum durations for operations, ensuring that slow or hung requests don't consume resources indefinitely. Timeouts are the primary trigger that feeds failure counts into circuit breaker state machines.

• Connection timeout: Max time to establish a TCP connection (typically 2-5s)
• Request timeout: Max time to receive a complete response (typically 10-30s)
• Idle timeout: Max time a connection can remain inactive

Without proper timeouts, a circuit breaker cannot detect failures quickly enough to prevent resource exhaustion. Timeout values must be tuned per dependency based on observed p95 and p99 latency percentiles.