Circuit Breaker Pattern

The core of the pattern is a state machine with three distinct states that govern how requests to a failing service are handled:

• Closed: The normal operating state. Requests pass through to the service. Failures are tracked, and if a failure threshold is exceeded, the breaker trips and moves to the Open state.
• Open: The protective state. All requests immediately fail fast without attempting the operation. A timer is set, after which the breaker moves to the Half-Open state.
• Half-Open: The probing state. A limited number of test requests are allowed to pass. Success resets the breaker to Closed; failure returns it to Open.

The primary benefit is preventing cascading failures and resource exhaustion. When the breaker is Open:

• The calling service receives an immediate, predictable error (e.g., "Service Unavailable").
• This allows the caller to implement graceful degradation, such as:
  • Returning cached or default data.
  • Queuing requests for later retry.
  • Failing over to an alternative service.
• It prevents threads, connections, or memory from being tied up waiting for timeouts from a failing dependency.

The breaker's behavior is tuned through key parameters that define its sensitivity and recovery timing:

• Failure Threshold: The count or percentage of recent requests that must fail (e.g., 5 failures in 10 seconds) to trip the breaker from Closed to Open.
• Timeout Duration: The length of time the breaker stays in the Open state before allowing a test request (Half-Open). This gives the failing service time to recover.
• Half-Open Request Limit: The number of test requests allowed in the Half-Open state before deciding the service's health (often just 1).
• Sliding Window: Many implementations use a time-based window to count failures, ensuring old failures don't indefinitely affect the state.

The Circuit Breaker is often combined with other resilience patterns to form a robust strategy:

• Retry Pattern: Used inside the Closed state. Transient failures are retried with exponential backoff before being counted as a failure by the breaker.
• Fallback Mechanism: When the breaker is Open or a call fails, a predefined fallback routine executes. This could be a static response, a value from a local cache, or a call to a secondary, less-capable service.
• Bulkhead Pattern: Used alongside circuit breakers to isolate dependencies into separate resource pools (thread pools, connections). This prevents one tripped breaker from consuming all resources and affecting other operations.

Effective circuit breakers provide rich telemetry, which is critical for orchestration observability in multi-agent systems:

• State Transitions: Logging events for every state change (CLOSED → OPEN, OPEN → HALF-OPEN).
• Failure Rates: Metrics on request counts, success rates, and latency percentiles.
• Health Status: Exposing the breaker's current state (e.g., via a health check API) allows orchestration engines to make routing decisions.
• This telemetry feeds into dashboards and alerts, enabling operators to identify chronically failing services and understand system health.

In agent orchestration, circuit breakers manage inter-agent dependencies to prevent a single failing agent from destabilizing the entire workflow.

• Agent-to-Agent Calls: An agent calling another agent's capability (e.g., a Planner agent requesting data from a Retrieval agent) should be wrapped in a circuit breaker.
• Tool/API Execution: When agents call external tools or APIs, a circuit breaker protects the agent from hanging indefinitely on a downstream failure.
• Orchestrator-Level Protection: The central workflow engine can use circuit breakers on critical agent pools, automatically rerouting tasks if a specific agent type becomes unhealthy, enabling self-healing system behaviors.