Circuit Breaker Pattern

When the circuit is Open or a call times out, a fallback strategy is invoked to provide a graceful degradation of service instead of a complete failure. Common fallbacks include:

• Returning a cached, stale value.
• Providing a default or empty response.
• Delegating the request to a secondary, less optimal service.
• Returning a user-friendly error message. This decouples the client's stability from the dependency's health.

Circuit breakers and retries are complementary patterns but must be coordinated to avoid contention.

• Retries are for transient, momentary failures (e.g., network blip).
• Circuit Breakers are for persistent, longer-lasting failures (e.g., downstream service crash). Best practice is to implement retries with exponential backoff inside the Closed state. Once the circuit trips to Open, retries cease immediately, preventing wasteful load on the failing system.

The state of circuit breakers is a primary health metric for distributed systems. Effective implementation requires exposing:

• State Transitions: Logs or events for every change (Closed → Open, Open → Half-Open, etc.).
• Request Metrics: Counts of successful, failed, and short-circuited (rejected) calls.
• Latency Percentiles: To help tune timeout values. This telemetry is crucial for SREs and DevOps teams to diagnose systemic issues and tune circuit parameters.

This is the pattern's primary purpose. In a microservices architecture, a slow or failing Service B can cause Service A's threads to block while waiting for a response. If traffic to A remains high, it can exhaust its own resources (threads, memory), causing it to fail—a cascade. The circuit breaker fails fast by immediately rejecting calls to B when it's unhealthy, preserving A's resources and overall system stability. It acts as a bulkhead between services.

A Dead Letter Queue is a holding queue for messages or tasks that cannot be processed successfully after repeated attempts. It works in concert with Circuit Breakers in message-driven architectures.

• Failure Finalization: When a Circuit Breaker is open and retries are halted, problematic messages can be routed to a DLQ for manual inspection or automated replay later.
• Audit Trail: DLQs provide a durable audit log of failures, aiding in debugging and ensuring no work is silently lost. This is crucial for financial or order-processing workflows where every transaction must be accounted for.

A Fallback Mechanism provides an alternative course of action when a primary operation fails and a Circuit Breaker is open. It enables graceful degradation of service.

• Static Response: Returning cached data, default values, or a simplified, non-personalized response.
• Alternative Service Path: Routing the request to a secondary, possibly less accurate or more expensive, service (e.g., a different LLM provider, a legacy API).
• Queue for Later Processing: Placing the request in a queue to be retried when the Circuit Breaker resets, allowing the user to continue without blocking.

A Health Check is a diagnostic probe used to determine the operational status of a service. It is often the mechanism a Circuit Breaker uses during its half-open state to test if the underlying issue has resolved.

• Liveness Probe: Determines if the service is running. A failed liveness check would keep the Circuit Breaker open.
• Readiness Probe: Determines if the service is ready to accept traffic (e.g., dependencies are initialized).
• Synthetic Transaction: A lightweight, non-critical request sent by the Circuit Breaker to verify full functionality before closing and resuming normal traffic.

Rate Limiting controls the number of requests a client can make to a service in a given time window. It is a proactive cousin to the reactive Circuit Breaker.

• Prevention vs. Reaction: Rate limiting prevents overload, while a Circuit Breaker reacts to it. They are often used together.
• Different Triggers: Rate limiting is triggered by request volume exceeding a quota, regardless of success/failure. A Circuit Breaker is triggered by a high rate of failures, regardless of request volume.
• Client-Side vs. Server-Side: Circuit Breakers are typically implemented on the client side (calling service), while rate limits are enforced by the server (called service).