Composite SLI

A Composite SLI is not a simple average but a weighted or formulaic combination of constituent SLIs. Common aggregation functions include:

• Weighted sums (e.g., 0.6 * Planning Success Rate + 0.4 * Action Success Ratio)
• Minimum or maximum functions to track the worst/best-performing component
• Geometric means for rate-based metrics
• Conditional logic (e.g., a composite fails if any critical guardrail SLI is breached). This allows engineering teams to define a single score that reflects the multi-dimensional nature of agent health.

Its primary purpose is to provide a unified view of a complex capability by combining related but distinct operational signals. For example:

• An Overall Efficiency Score could combine Task Completion Rate, End-to-End Task Latency, and Redundant Action Ratio.
• A Safety & Compliance Score might aggregate Guardrail Compliance Rate, Hallucination Rate, and Fallback Success Rate.
• A Resiliency Score could be derived from Self-Correction Success Rate, Retry Success Rate, and Health Check Success Rate. This moves monitoring from isolated metrics to actionable, business-aligned scores.

A Composite SLI is built upon well-defined, atomic Agentic SLIs. It does not measure raw telemetry but combines higher-level indicators. Prerequisites include:

• Established baselines for each constituent SLI (e.g., Planning Success Rate, Action Success Ratio).
• Clear understanding of the relationships and trade-offs between the underlying metrics.
• Reliable data pipelines for each component SLI. This derivation ensures the composite is traceable and debuggable; a drop in the composite score can be investigated by drilling into its atomic components.

Composite SLIs enable the definition of sophisticated Service Level Objectives (SLOs) for entire agent capabilities. Instead of managing dozens of individual SLOs, teams can set a target for the composite. For instance:

• "Our agent's Safety & Compliance Score must be ≥ 99.5% over a 30-day window."
• "The Overall Efficiency Score for the procurement agent must not drop below 0.85." This simplifies error budget calculation and consumption tracking for complex systems, as there is one primary budget for the composite behavior.

A shifting Composite SLI value serves as a high-level alert that a complex aspect of agent performance is degrading. It directs engineering attention before individual component SLIs breach their thresholds. The composite's structure informs triage:

• A drop in an Efficiency Score immediately points to planning, execution, or latency sub-systems.
• A decline in a Safety Score prioritizes checks on guardrails, fact-checking, and fallback mechanisms. This transforms observability from reactive monitoring to progressive system management.

A practical example is an Agent Reliability Index for a customer support agent, defined as: (0.3 * Task Completion Rate) + (0.3 * Result Accuracy) + (0.2 * Guardrail Compliance Rate) + (0.2 * (1 - Normalized Latency))

• Task Completion Rate: Ensures the agent finishes the job.
• Result Accuracy: Ensures the answer is correct.
• Guardrail Compliance Rate: Ensures the agent stays within policy.
• Normalized Latency: Penalizes slow performance. This single index, tracked over time, gives a CTO a clear, quantitative measure of the agent's end-to-end service quality.

An Agentic SLI is a quantitative measure of a specific aspect of an autonomous agent's performance. It is the fundamental building block for observability.

• Examples: Planning Success Rate, End-to-End Task Latency, Action Success Ratio.
• Purpose: Provides a precise, numerical signal of health for a single dimension of agent behavior.
• Usage: Tracked in real-time and aggregated over time to compute SLO compliance.

An Agentic SLO is a target value or range for an Agentic SLI, defining the acceptable level of performance over a specified period.

• Structure: Often expressed as a percentage (e.g., "Planning Success Rate ≥ 99.5% over 30 days").
• Function: Creates a formal reliability contract for the agent system.
• Error Budget: The allowable deviation from the SLO, calculated as (100% - SLO%) * time_window. This budget manages the trade-off between innovation velocity and system stability.

An Error Budget is the allowable amount of time an autonomous agent system can fail to meet its SLOs within a defined compliance period.

• Calculation: Derived directly from the SLO. For a 99.9% monthly SLO, the error budget is 0.1% of the month (~43.2 minutes).
• Operational Use: Serves as a shared resource between development and operations teams. Consuming the budget triggers a focus on reliability; preserving it allows for riskier feature deployments.
• Burn Rate: Measures how quickly the error budget is being consumed, a key signal for prioritizing incident response.

Planning Success Rate is a core Agentic SLI that measures the percentage of times an agent successfully decomposes a high-level goal into a valid, executable sequence of sub-tasks.

• Measurement Point: Evaluated after the agent's initial reasoning/planning phase, before execution begins.
• Importance: A low rate indicates fundamental flaws in the agent's reasoning capabilities or understanding of its action space.
• Example: An agent with a 98% Planning Success Rate fails to create a coherent plan for 2 out of every 100 tasks received.

End-to-End Task Latency is an Agentic SLI that measures the total clock time from when an agent receives a task to when it delivers a final, validated result.

• Scope: Includes all phases: planning, tool execution, waiting for external APIs, reflection, and final output generation.
• Key Consideration: For composite tasks, this is the primary user-facing latency metric, more critical than the latency of individual sub-steps.
• SLO Example: "95% of agent tasks complete within 10 seconds."

Self-Correction Success Rate is an Agentic SLI that measures the effectiveness of an agent's recursive error correction loops in identifying and remediating its own failures without human intervention.

• Mechanism: Tracks attempts where an agent, upon detecting a failure (e.g., a tool error, an invalid result), successfully adjusts its plan and retries to achieve the goal.
• Value: A high rate indicates robust resilience and reduces operational burden. It is a key component of a Resiliency Score.
• Failure Mode: A low rate may necessitate more aggressive fallback mechanisms or human-in-the-loop escalation.