Resiliency Score

This core SLI measures the percentage of times an agent successfully identifies and remediates its own errors through recursive loops without human intervention. A high rate indicates robust internal error handling.

• Mechanism: Tracks outcomes of reflection and replanning cycles.
• Impact on Score: Directly increases the Resiliency Score, as it reflects autonomous fault tolerance.
• Example: An agent failing a tool call, diagnosing the error (e.g., malformed parameters), and successfully retrying with a corrected payload.

This SLI measures the effectiveness of contingency logic, calculating the percentage of times an agent successfully executes an alternative path when its primary method fails. It validates the robustness of failover designs.

• Mechanism: Monitors switches to predefined backup tools, models, or workflows.
• Impact on Score: A high rate significantly bolsters the score, demonstrating graceful degradation.
• Example: An LLM-based agent switching from a primary high-latency model to a faster, less capable model when SLOs for response time are at risk.

This SLI quantifies the effectiveness of automatic retry logic for transient failures, calculated as the percentage of retried operations that ultimately succeed. It distinguishes between transient and permanent faults.

• Mechanism: Evaluates success after a configured number of retries with optional backoff or parameter adjustment.
• Impact on Score: A moderate positive impact; high success indicates resilience to flaky external dependencies.
• Example: An agent retrying a failed API call to a payment gateway, succeeding on the second attempt after a network timeout.

This safety-focused SLI measures the percentage of an agent's actions and outputs that adhere to predefined operational, safety, and ethical policy constraints. Resiliency includes maintaining safe operation under stress.

• Mechanism: Checks outputs and planned actions against a rule engine or safety classifier.
• Impact on Score: Non-compliance can severely degrade or nullify the score, as unsafe operation is a critical failure mode.
• Example: An agent in a financial context rejecting a user request that would violate compliance rules, even if technically executable.

This availability SLI measures the percentage of periodic diagnostic probes (liveness and readiness checks) that pass, indicating the agent's operational stability and preparedness to accept tasks.

• Mechanism: Runs synthetic transactions or internal state checks at regular intervals.
• Impact on Score: A foundational component; consistently failing health checks indicates systemic instability, lowering the overall score.
• Example: A readiness check verifying that an agent's memory vector store is connected and its core planning model is responsive.

The final Resiliency Score is calculated by applying defined weights to the normalized values of its constituent SLIs, then combining them (e.g., weighted average). This reflects organizational priorities.

• Normalization: Individual SLI values (often percentages) are scaled to a common range (e.g., 0-1).
• Weighting: Critical SLIs like Guardrail Compliance Rate carry more weight than others like Retry Success Rate.
• Output: A single numerical score (e.g., 0-100 or 0-1) providing an at-a-glance assessment of agent robustness.

A core input to the Resiliency Score. This Agentic SLI measures the percentage of times an autonomous agent successfully identifies and remediates its own errors through recursive loops without human intervention. It is a direct indicator of an agent's internal fault tolerance and capacity for self-healing.

• Key Mechanism: Often involves reflection steps where the agent critiques its own plan or output and generates a corrected version.
• Impact on Resiliency: A high rate suggests the agent can recover from many internal logic or planning failures, reducing the need for external fallbacks.

A critical component of the Resiliency Score. This Agentic SLI measures the percentage of times an agent successfully invokes a predefined contingency plan or alternative execution path when its primary method fails. It quantifies the effectiveness of redundant systems and graceful degradation strategies.

• Operational Role: Activated when primary tools are unavailable, responses are invalid, or guardrails are triggered.
• Example: An agent failing to book a flight via a primary API automatically and successfully switching to a secondary provider or a human-in-the-loop queue.

The formal category for metrics like the Resiliency Score. A Composite SLI is a Service Level Indicator derived from the mathematical combination (e.g., weighted average, minimum) of two or more underlying SLIs. It provides a unified, high-level signal for complex system qualities.

• Purpose: Simplifies monitoring and SLO definition for multifaceted attributes like safety, efficiency, or resiliency.
• Construction: A Resiliency Score might be computed as (0.4 * Self-Correction Success Rate) + (0.4 * Fallback Success Rate) + (0.2 * Retry Success Rate).

Often a secondary input to resiliency calculations. This Agentic SLI measures the effectiveness of an agent's automatic retry logic for transient failures, calculated as the percentage of retried operations (e.g., API calls, tool executions) that ultimately succeed.

• Distinction from Fallback: Retries attempt the same action, often with exponential backoff, while fallbacks switch to a different method.
• Resiliency Contribution: A high rate indicates robust handling of network flakiness and temporary external service unavailability, preventing unnecessary escalation to fallback paths.

The operational framework that gives a Resiliency Score its business context. An Error Budget is the allowable amount of time a system can fail to meet its SLOs within a compliance period. It quantifies the trade-off between reliability and innovation velocity.

• Usage: If the Resiliency Score (tied to an SLO) is consistently high, the error budget is preserved, allowing for riskier deployments or feature launches.
• Management: A declining Resiliency Score consumes the error budget, potentially triggering a release freeze or reliability investment period.

The target that a Resiliency Score is measured against. An Agentic SLO is a target value or range for an Agentic SLI (like a Resiliency Score), defining the acceptable level of performance over a specified period.

• Relationship: A team might define an SLO such as "Resiliency Score ≥ 99.5% over a 30-day rolling window."
• Enforcement: Breaching this SLO consumes the Error Budget. The Resiliency Score is the primary SLI used to track compliance with this resiliency-focused SLO.