Circuit Breaker

The core logic of a circuit breaker is a finite state machine with three distinct states:

• CLOSED: Normal operation. Requests pass through to the dependent service while failures are counted.
• OPEN: The circuit has 'tripped'. Requests fail immediately without reaching the unhealthy service, returning a fallback response or error.
• HALF-OPEN: A probationary state. After a timeout, a single test request is allowed. Its success resets the circuit to CLOSED; its failure returns it to OPEN. This stateful design prevents the system from repeatedly attempting doomed requests, allowing the failing service time to recover.

The transition from CLOSED to OPEN is governed by configurable thresholds that define 'failure'.

• Failure Count/Percentage: The circuit trips after a defined number of consecutive failures (e.g., 5) or a percentage of recent requests (e.g., 50% over the last 100 calls).
• Timeout Duration: Defines what constitutes a slow call, which can be counted as a failure.
• Sliding Window: Failures are typically tracked within a recent time window (e.g., last 60 seconds) to ensure the circuit responds to current conditions, not stale history. These parameters allow fine-tuning for the specific latency and reliability profile of the external dependency.

When the circuit is OPEN, calls must not reach the failing service. The system requires a defined fallback strategy to maintain partial functionality. Common strategies include:

• Static Response: Return a cached, default, or stale value.
• Alternative Service: Route the request to a backup or degraded service.
• Queue for Later: Place the request in a dead-letter queue for asynchronous retry.
• Informative Error: Return a clear, immediate error (e.g., 503 Service Unavailable). This enables graceful degradation, ensuring the user experience degrades predictably instead of failing catastrophically.

The HALF-OPEN state is the circuit breaker's self-healing mechanism. After a configured reset timeout in the OPEN state, it allows a single 'probe' request to pass.

• Success: If the probe succeeds, the circuit assumes the dependency is healthy and resets to CLOSED.
• Failure: If the probe fails, the circuit returns to OPEN, and the reset timer restarts. This automated recovery loop eliminates the need for manual intervention after transient outages, making the system resilient to intermittent failures.

A production-grade circuit breaker must emit detailed telemetry for observability. Key metrics include:

• Circuit State Changes: Logs for transitions between CLOSED, OPEN, and HALF-OPEN.
• Request Counts: Successes, failures (timeouts, exceptions), and short-circuited calls (rejected while OPEN).
• Latency Percentiles: Performance of calls through the circuit. This data is critical for Service Level Objective (SLO) tracking, alerting on persistent failures, and tuning the breaker's configuration parameters based on real-world behavior.

The primary purpose of a circuit breaker is to isolate failures and prevent cascading failures across a distributed system. Without it:

• Threads/Connections pool exhaustion: Waiting clients exhaust resources.
• Increased latency: Backlogs cause slowdowns for all users.
• Resource saturation: Memory and CPU spike on the calling service. By failing fast and opening the circuit, the pattern protects the calling service's resources, allowing it to continue serving traffic for other, healthy functionalities. This containment is fundamental to building bulkheads in a microservices or edge AI architecture.

A predefined alternative course of action when a primary service or operation fails. It is the action taken after a circuit breaker opens.

• Common Fallbacks for Edge AI:
  • Return a cached response.
  • Use a simplified, less accurate local model.
  • Provide a default or degraded response.
  • Queue the request for later processing.
• Design Goal: Maintain partial or graceful functionality instead of a complete failure, preserving user experience even when dependencies are unhealthy.