Circuit Breaker Pattern

The pattern's core logic is defined by a state machine with three states:

• Closed: The normal operating state. Requests flow through, and failures are counted.
• Open: The circuit has 'tripped.' All requests fail immediately without attempting the operation, allowing the failing system time to recover.
• Half-Open: A trial state after a timeout. A limited number of test requests are allowed. Success moves the circuit back to Closed; failure returns it to Open.

The circuit monitors for failures to decide when to trip. Key configurable parameters include:

• Failure Threshold: The count (e.g., 5) or percentage (e.g., 50%) of recent calls that must fail to transition from Closed to Open.
• Sliding Time Window: Failures are typically counted within a recent time window (e.g., the last 60 seconds) to prevent stale failures from affecting the state.
• Timeout Duration: The length of time the circuit remains Open before transitioning to Half-Open for a health check.

When the circuit is Open or a call times out, a fallback strategy is invoked to provide a graceful degradation of service instead of a complete failure. Common fallbacks include:

• Returning a cached, stale value.
• Providing a default or empty response.
• Delegating the request to a secondary, less optimal service.
• Returning a user-friendly error message. This decouples the client's stability from the dependency's health.

Circuit breakers and retries are complementary patterns but must be coordinated to avoid contention.

• Retries are for transient, momentary failures (e.g., network blip).
• Circuit Breakers are for persistent, longer-lasting failures (e.g., downstream service crash). Best practice is to implement retries with exponential backoff inside the Closed state. Once the circuit trips to Open, retries cease immediately, preventing wasteful load on the failing system.

The state of circuit breakers is a primary health metric for distributed systems. Effective implementation requires exposing:

• State Transitions: Logs or events for every change (Closed → Open, Open → Half-Open, etc.).
• Request Metrics: Counts of successful, failed, and short-circuited (rejected) calls.
• Latency Percentiles: To help tune timeout values. This telemetry is crucial for SREs and DevOps teams to diagnose systemic issues and tune circuit parameters.

This is the pattern's primary purpose. In a microservices architecture, a slow or failing Service B can cause Service A's threads to block while waiting for a response. If traffic to A remains high, it can exhaust its own resources (threads, memory), causing it to fail—a cascade. The circuit breaker fails fast by immediately rejecting calls to B when it's unhealthy, preserving A's resources and overall system stability. It acts as a bulkhead between services.

Retry logic is an error-handling strategy where a failed operation is automatically re-executed after a delay. It is the primary mechanism the Circuit Breaker monitors and controls.

• Exponential Backoff: A common policy that increases the wait time between retries exponentially (e.g., 1s, 2s, 4s, 8s) to avoid overwhelming a struggling service.
• Jitter: Randomization added to backoff intervals to prevent synchronized retry storms from multiple clients.
• Max Attempts: A critical limit to prevent infinite retry loops. The Circuit Breaker often acts as a higher-level governor when retry limits are exhausted.

The Bulkhead Pattern isolates elements of an application into pools, so if one fails, the others continue to function. It is a complementary pattern to the Circuit Breaker for containing failures.

• Resource Isolation: Prevents a single point of failure from cascading and consuming all system resources (e.g., thread pools, connections).
• Parallel to Circuit Breaker: While a Circuit Breaker stops calls to a failing service, a Bulkhead ensures the failure doesn't block calls to healthy services. For example, isolating database calls for user authentication from product catalog queries.

A Dead Letter Queue is a holding queue for messages or tasks that cannot be processed successfully after repeated attempts. It works in concert with Circuit Breakers in message-driven architectures.

• Failure Finalization: When a Circuit Breaker is open and retries are halted, problematic messages can be routed to a DLQ for manual inspection or automated replay later.
• Audit Trail: DLQs provide a durable audit log of failures, aiding in debugging and ensuring no work is silently lost. This is crucial for financial or order-processing workflows where every transaction must be accounted for.

A Fallback Mechanism provides an alternative course of action when a primary operation fails and a Circuit Breaker is open. It enables graceful degradation of service.

• Static Response: Returning cached data, default values, or a simplified, non-personalized response.
• Alternative Service Path: Routing the request to a secondary, possibly less accurate or more expensive, service (e.g., a different LLM provider, a legacy API).
• Queue for Later Processing: Placing the request in a queue to be retried when the Circuit Breaker resets, allowing the user to continue without blocking.

A Health Check is a diagnostic probe used to determine the operational status of a service. It is often the mechanism a Circuit Breaker uses during its half-open state to test if the underlying issue has resolved.

• Liveness Probe: Determines if the service is running. A failed liveness check would keep the Circuit Breaker open.
• Readiness Probe: Determines if the service is ready to accept traffic (e.g., dependencies are initialized).
• Synthetic Transaction: A lightweight, non-critical request sent by the Circuit Breaker to verify full functionality before closing and resuming normal traffic.

Rate Limiting controls the number of requests a client can make to a service in a given time window. It is a proactive cousin to the reactive Circuit Breaker.

• Prevention vs. Reaction: Rate limiting prevents overload, while a Circuit Breaker reacts to it. They are often used together.
• Different Triggers: Rate limiting is triggered by request volume exceeding a quota, regardless of success/failure. A Circuit Breaker is triggered by a high rate of failures, regardless of request volume.
• Client-Side vs. Server-Side: Circuit Breakers are typically implemented on the client side (calling service), while rate limits are enforced by the server (called service).