Circuit Breaker

The circuit breaker continuously tracks a configurable failure rate over a sliding time window. When the ratio of failed tool calls, invalid outputs, or policy violations exceeds a predefined threshold (e.g., 50% failure rate over 30 seconds), the breaker trips to an open state, immediately halting all downstream agent actions. This prevents a single malfunctioning agent from consuming resources, corrupting data, or triggering destructive cascades across interconnected systems. Common threshold metrics include consecutive failure count, error rate percentage, and latency degradation.

Unlike a simple binary on/off switch, a robust circuit breaker implements a half-open state for controlled recovery. After a configurable cooldown period (e.g., 60 seconds), the breaker permits a limited number of probe requests to test if the underlying failure condition has resolved. If these probes succeed, the breaker resets to a closed state; if they fail, it immediately returns to open. This pattern prevents thrashing—the rapid oscillation between open and closed states—and allows the system to self-heal without manual intervention.

When a circuit breaker trips, the agent must not simply crash—it must degrade gracefully. This involves predefined fallback behaviors:

• Static fallback: Return a cached or default safe response instead of live generation
• Function stripping: Disable only the failing tool or capability while maintaining core agent functionality
• Escalation routing: Automatically hand off the task to a human operator or a more conservative backup model
• State preservation: Serialize the agent's current context and memory so the task can resume cleanly after recovery This ensures the user experiences reduced functionality rather than a complete service outage.

Circuit breakers are often deployed alongside the bulkhead pattern to limit blast radius. In agentic systems, this means partitioning tool access, memory stores, and execution threads into isolated pools. If one agent instance or tool category triggers a circuit breaker, only that specific partition is affected—other agent capabilities continue operating normally. For example, a breaker on a web search tool should not disable the agent's code execution or database query capabilities. This requires careful namespace segmentation and per-tool breaker instances.

Every state transition in a circuit breaker must emit structured telemetry for real-time observability. Critical events include:

• Trip events: Timestamp, failure count, threshold breached, and the specific tool or agent identifier
• Probe outcomes: Success/failure of half-open recovery attempts
• Manual resets: Audit trail of human operator overrides These events feed into monitoring dashboards and alerting systems, enabling SRE teams to detect systemic issues—such as upstream API degradation or model drift—before they trigger widespread breaker cascades across an agent fleet.

Advanced agentic circuit breakers incorporate contextual awareness into their tripping decisions. Rather than using a single global threshold, they evaluate failure severity based on:

• Action criticality: A failed financial transaction warrants immediate tripping; a failed weather lookup may not
• User role: Actions initiated by privileged users may have higher tolerance thresholds
• Task state: Failures during idempotent operations are less severe than failures in non-reversible state changes
• Cost accumulation: Breakers can trip when cumulative API costs exceed a budget, preventing financial runaway loops This nuanced approach avoids false positives while maintaining strict safety guarantees.

A control mechanism that restricts the number of actions or API calls an agent can make within a specific time window. Unlike a circuit breaker—which trips based on failure rates—a rate limiter enforces a hard ceiling on throughput to prevent resource exhaustion, runaway loops, or financial drain. Common implementations include token bucket, leaky bucket, and sliding window algorithms. Rate limiters operate at the pre-execution stage, blocking requests before they reach the target system, while circuit breakers monitor outcomes and react to degradation.

A control point in an agentic workflow that requires explicit validation or approval before a high-stakes tool call or state-changing operation is executed. Action gates serve as human-in-the-loop checkpoints for operations that exceed a defined risk threshold—such as deleting production data, sending external communications, or executing financial transactions. While circuit breakers are automated and reactive, action gates are deliberative and preventive, introducing a synchronous approval step that pauses the agent's execution until authorization is granted.

A minimum probability score that an agent's output or classification must exceed to be considered valid. Outputs falling below this threshold are rejected or flagged for review. Confidence thresholds act as a soft circuit breaker at the decision level—when the model's certainty drops below a calibrated boundary, the system refuses to act rather than proceeding with low-quality output. Key implementation considerations include threshold calibration against domain-specific error costs and the trade-off between false positives (rejecting good outputs) and false negatives (accepting bad ones).

The practice of enforcing strict limits on the total number of input and output tokens an agent can consume per task or session. Token budgets prevent context window overflow attacks, control inference costs, and bound latency. When a budget is exhausted, the system triggers a graceful degradation—summarizing state, requesting human intervention, or terminating the session. This functions as a resource-level circuit breaker, halting execution when computational expenditure exceeds predefined economic or operational boundaries.

A security mechanism that runs an agent's generated code or tool calls inside an isolated, ephemeral environment—such as a Docker container, gVisor sandbox, or WebAssembly runtime—to prevent access to the host system. Sandboxing complements circuit breakers by ensuring that even when an agent acts erroneously before a breaker trips, the blast radius is contained. Key properties include filesystem isolation, network egress controls, and mandatory resource caps that terminate processes exceeding memory or CPU limits.

A programmatic policy or mechanism that constrains an AI agent's behavior to prevent harmful, off-policy, or unsafe actions and outputs. Guardrails are the broader category that encompasses circuit breakers, content filters, and action gates. They operate at multiple layers: input guardrails validate prompts before processing, output guardrails sanitize generated content, and behavioral guardrails—like circuit breakers—monitor execution patterns. Frameworks such as NVIDIA NeMo Guardrails and Guardrails AI provide programmable interfaces for defining these constraints.