Self-Correction Success Rate

The initial mechanism that flags a failure requiring correction. This is not a simple HTTP error code but a semantic evaluation of the agent's own output or state.

• Internal Validation: The agent runs its output against predefined rules, a verifier model, or a formal specification.
• External Feedback: The environment (e.g., an API response, a user signal, a monitoring check) provides a negative reward or explicit error.
• Self-Critique Loop: A dedicated reasoning step where the agent assesses the quality, safety, or feasibility of its proposed action before or after execution.

The observability hooks that capture the agent's recursive attempt to fix the detected error. This measures the process of correction.

• Loop Counter: Tracks the number of recursive correction attempts for a single root task.
• State Delta Monitoring: Compares the agent's internal state (goal, plan, context) before and after a correction cycle to confirm a meaningful adjustment was made.
• Path Divergence: Measures how significantly the new execution plan deviates from the failed one, ensuring the correction isn't a trivial retry.

The precise, binary conditions that determine if a correction attempt is counted as a success. This is the most critical and nuanced component.

• Task Completion: The ultimate goal is achieved after the correction, even if via a different path.
• Error Resolution: The specific condition that triggered the error detection is no longer present (e.g., a malformed API call is now valid).
• Constraint Satisfaction: The new solution adheres to all original guardrails, cost limits, and policy requirements that the failed attempt violated.

Defines the time window and task scope for which a correction is considered valid, preventing metric inflation from unrelated successes.

• Session Boundary: Corrections are only counted if they occur within the same logical agent session or task episode.
• Time-to-Correct Limit: A correction must be generated and executed within a defined SLA (e.g., 5 seconds) to count as a successful self-correction, not a new task.
• Context Preservation: The correction must operate on the same core objective and contextual facts; a total task reset is a failure, not a correction.

The mathematical definition of the metric, which synthesizes the instrumented components into a single percentage.

Standard Formula: Self-Correction Success Rate = (Number of Detected Errors Successfully Corrected) / (Total Number of Detected Errors Requiring Correction) * 100

Key Nuances:

• The denominator excludes errors where the agent correctly invoked a human-in-the-loop fallback, as this is a designed failure mode.
• A multi-loop correction for one error counts as one success if the final loop succeeds, but the loop count is a related diagnostic metric.

Secondary metrics that provide context for interpreting the primary success rate and diagnosing failures.

• Mean Corrections Per Error: The average number of recursive loops needed to resolve a detected error. A high value indicates struggling correction logic.
• Correction Latency: The time delta between error detection and successful correction completion.
• Correction Path Efficiency: Measures the resource cost (tokens, API calls) of the successful correction path versus the original failed path.
• Detection False Negative Rate: The percentage of ultimate task failures that were not preceded by an internal error detection, indicating blind spots in the agent's self-awareness.

An Agentic SLI is a quantitative measure of a specific aspect of an autonomous agent's performance. Unlike traditional SLIs for APIs, these indicators capture the unique behaviors of reasoning systems.

• Examples: Planning Success Rate, Task Completion Latency, Hallucination Rate.
• Purpose: Provides the raw, measurable data used to define Service Level Objectives (SLOs) and assess operational health.

An Agentic SLO is a target value or range for an Agentic Service Level Indicator, defining the acceptable level of performance for an autonomous agent system.

• Example: "Self-Correction Success Rate ≥ 95% over a 30-day rolling window."
• Function: SLOs create a formal contract for reliability, enabling data-driven decisions about deploying new features or agent versions based on the remaining Error Budget.

Fallback Success Rate measures the percentage of times an autonomous agent successfully invokes a predefined contingency plan when its primary execution path fails. It is a complementary SLI to Self-Correction Success Rate.

• Key Difference: Self-correction involves dynamic, recursive error recovery, while fallback mechanisms are typically static, pre-programmed alternative paths.
• Use Case: An agent failing to call a primary API might fall back to a cached response or a different service provider.

Retry Success Rate measures the effectiveness of an agent's automatic retry logic for failed actions, calculated as the percentage of retried operations that ultimately succeed. It is a lower-level component often feeding into the broader Self-Correction Success Rate.

• Mechanism: Tracks simple, immediate re-execution of a failed tool call or API request, often with exponential backoff.
• Contrast: Self-correction may involve re-planning, choosing a different tool, or gathering more context, not just retrying the same action.

A Resiliency Score is a composite metric derived from multiple SLIs—such as Self-Correction Success Rate, Fallback Success Rate, and Health Check Success Rate—that quantifies an autonomous agent's overall ability to maintain functionality despite errors or external failures.

• Calculation: Often a weighted formula combining key resilience indicators into a single, actionable number (e.g., 0-100).
• Business Value: Provides CTOs and engineering leaders with a high-level view of system robustness and fault tolerance.

An Error Budget is the allowable amount of time an autonomous agent system can fail to meet its SLOs within a defined period. It is calculated from the SLO target (e.g., 99.9% success) and represents the "budget" for unreliability.

• Formula: Error Budget = (1 - SLO) * Measurement Period. For a 95% monthly SLO, the error budget is 5% of the month (~36 hours).
• Operational Use: Consuming the error budget too quickly (high SLO Burn Rate) triggers a freeze on risky deployments until reliability is restored.