Circuit Breaker Pattern

The core logic of a circuit breaker is modeled as a finite state machine with three distinct states:

• CLOSED: The normal operating state. Requests flow through to the dependency. Failures are counted.
• OPEN: The circuit has 'tripped.' All requests to the dependency fail immediately without attempting the call, implementing a fail-fast behavior.
• HALF-OPEN: After a configured timeout, the circuit allows a limited number of test requests through. Their success or failure determines if the circuit returns to CLOSED (recovered) or reverts to OPEN (still failing).

A circuit breaker transitions from CLOSED to OPEN based on threshold-based rules evaluated over a rolling time window. Common conditions include:

• Failure Rate Threshold: Trip if X% of recent calls (e.g., 50%) result in errors.
• Slow Call Rate Threshold: Trip if X% of calls exceed a latency threshold, treating slow responses as failures.
• Consecutive Failure Count: Trip after a specified number of failures in a row, useful for low-traffic services.
• Volume Threshold: A minimum number of calls within the window before tripping is considered, preventing false positives during low traffic.

The half-open state is the critical recovery mechanism. After the circuit has been OPEN for a defined sleep period, it transitions to HALF-OPEN and permits a small, configurable number of test requests (often just one) to pass through.

• Success: If the probe request(s) succeed, the circuit assumes the underlying issue is resolved and transitions to CLOSED, resuming normal operation.
• Failure: If the probe fails, the circuit immediately transitions back to OPEN, restarting the sleep timer. This prevents a recovering but unstable service from being flooded.

When a circuit is OPEN or a call fails, the system should not simply crash. A robust implementation provides a fallback mechanism, enabling graceful degradation. Fallback options include:

• Returning a cached, stale, or default value.
• Calling an alternative, less-capable service.
• Returning a user-friendly error message indicating temporary unavailability.
• Queuing the request for later retry. This allows the system to maintain partial functionality while the faulty dependency is offline.

Circuit breakers are a primary source of operational telemetry. They should emit clear, actionable metrics and events for monitoring systems, including:

• State transition events (CLOSED → OPEN, OPEN → HALF-OPEN, etc.).
• Current state gauges and failure rate counters.
• NotCallPermittedException counts (requests rejected while OPEN). This data is essential for alerting teams to systemic failures and for conducting post-incident analysis and chaos engineering experiments.

The Circuit Breaker is rarely used in isolation. It is a foundational component within a broader resilience engineering toolkit and integrates with:

• Retry Logic with Exponential Backoff & Jitter: Used inside a CLOSED circuit for transient faults. The circuit breaker protects against retry storms.
• Bulkhead Pattern: Isolates different service calls into separate resource pools (e.g., thread pools). A circuit breaker can be applied per bulkhead, preventing a failure in one dependency from consuming all resources.
• Health Checks: Can be used as a probe mechanism for the HALF-OPEN state or to inform circuit breaker configuration.
• Load Shedding: Works in tandem; circuit breakers stop calls to a failing dependency, while load shedding rejects incoming traffic to protect the current service.

A resilience pattern that isolates elements of an application into independent pools of resources (threads, connections, instances). If one pool fails or becomes saturated, the failure is contained, preventing it from cascading and exhausting all resources in the system. This is analogous to the watertight compartments (bulkheads) in a ship.

• Key Mechanism: Resource isolation and quota management.
• Prevents: A single point of failure from consuming all connection pools or threads.
• Example: Separating database calls for a "checkout" service from calls for a "product recommendations" service into distinct thread pools.

A programming technique for handling transient faults by automatically re-attempting a failed operation. Exponential Backoff is a strategy where the wait time between retries increases exponentially (e.g., 1s, 2s, 4s, 8s). This is often used before a circuit breaker trips.

• Purpose: To handle temporary network glitches or brief service unavailability.
• Risk: Indiscriminate retries can overwhelm a struggling service, causing a thundering herd problem.
• Circuit Breaker Role: The breaker opens when retries consistently fail, moving the system into a fail-fast mode to stop the retry storm.

A Fallback is a predefined alternative action or response when a primary operation fails. Graceful Degradation is the system design principle of maintaining core functionality while reducing non-essential features under failure conditions.

• Circuit Breaker Synergy: When a circuit is open, the system immediately executes the fallback instead of attempting the likely-failing call.
• Fallback Examples: Returning cached data, static default values, a simplified user interface, or queuing a request for later processing.
• Goal: To provide a degraded but acceptable user experience while the underlying dependency recovers.

A Health Check is a periodic diagnostic request (e.g., GET /health) to verify a service's operational status. Outlier Detection is a mechanism, common in service meshes, that identifies and temporarily ejects unhealthy instances from a load balancing pool.

• Active vs. Passive: Health checks are active probes, while circuit breakers typically use passive monitoring of real request outcomes.
• Integration: A failing health check can be a signal to preemptively open a circuit. Outlier detection acts as a client-side circuit breaker at the load balancer level.
• Purpose: To ensure traffic is only routed to healthy, responsive service instances.

Chaos Engineering is the discipline of proactively experimenting on a system to build confidence in its resilience. Fault Injection is the deliberate introduction of failures (latency, errors, crashes) into a system to test its reactions.

• Primary Use: To validate that resilience patterns like Circuit Breaker, Retry, and Fallback work as designed under real failure conditions.
• Testing the Breaker: Engineers inject latency or HTTP 500 errors into a dependency to verify the circuit opens at the configured threshold and that fallbacks engage.
• Goal: To discover systemic weaknesses before they cause an unplanned outage.

Load Shedding is the proactive rejection of non-critical requests when a system is under excessive load. Backpressure is a flow control mechanism where a overwhelmed downstream service signals upstream callers to slow down or stop sending data.

• Relationship to Circuit Breaker: All are flow-control patterns. A circuit breaker stops calls to a failing dependency. Load shedding rejects excess traffic at the entry point to protect the system's own resources. Backpressure propagates saturation signals upstream.
• Circuit Breaker as Backpressure: An open circuit is a form of backpressure, signaling clients to stop sending requests until recovery.
• Combined Use: A system might use load shedding at its API gateway, internal circuit breakers for dependencies, and backpressure in streaming data pipelines.