Health Check Success Rate

Health Check Success Rate is the percentage of periodic diagnostic probes (liveness and readiness checks) against an autonomous agent that pass within a defined time window. Its primary purpose is to provide a binary, operational signal of whether the agent's core runtime is available to accept and process work, separate from its functional correctness on specific tasks.

• Liveness Probes: Confirm the agent process is running and responsive (e.g., the container or service host is alive).
• Readiness Probes: Verify the agent has all required dependencies (e.g., model endpoints, vector databases, API gateways) and is in a state to execute tasks. A low rate indicates the agent is frequently unavailable, making higher-level SLIs like Task Completion Rate irrelevant.

Implementation involves lightweight, synthetic requests that test the agent's minimum viable operational state. Key components include:

• Probe Endpoint: A dedicated API route (e.g., /health) that executes a minimal, deterministic check.
• Check Logic: Validates internal subsystems: session manager responsiveness, context window availability, connection to critical external dependencies (LLM API, tool registry).
• Timeout Configuration: Aggressive timeouts (e.g., < 1 second) to prevent the probe from masking latency issues.
• Frequency: High-frequency polling (e.g., every 10-30 seconds) to provide near-real-time availability signals. The probe must be cheap to execute and must not trigger actual business logic or consume significant context window tokens.

It is critical to differentiate Health Check Success Rate from SLIs that measure agent capability or quality. This SLI answers "Is it up?" not "Is it working well?"

• vs. Planning Success Rate: Health checks do not evaluate the agent's ability to decompose a novel goal; they only verify the planning subsystem is reachable.
• vs. Action Success Ratio: A health check may confirm the tool-calling framework is loaded, but does not execute real tool calls against production APIs.
• vs. End-to-End Task Latency: Health check latency is a separate, infrastructure-level metric; the SLI is based on pass/fail status, not speed. A 99.9% Health Check Success Rate with a 70% Task Completion Rate indicates an available but frequently failing agent, directing investigation to logic errors, not infrastructure.

This SLI is the primary input for availability-focused Service Level Objectives (SLOs). A common SLO is "Health Check Success Rate >= 99.95% over a 30-day rolling window."

• Error Budget Calculation: The SLO defines an allowable error budget (e.g., 0.05% failure). Exhausting this budget triggers a freeze on risky deployments.
• Burn Rate Monitoring: The speed at which the error budget is consumed. A rapid burn rate on health checks indicates a systemic infrastructure issue requiring immediate intervention.
• Composite SLIs: Health Check Success Rate is often a mandatory component of a Composite SLI for overall system health, combined with metrics like dependency latency.

A dip in Health Check Success Rate is a primary alerting signal for Site Reliability Engineers (SREs). Diagnostic steps following an alert include:

• Dependency Isolation: Checking the status of each subsystem validated by the readiness probe (LLM provider, database, memory store).
• Resource Analysis: Inspecting compute resource exhaustion (CPU, memory, GPU memory) on the agent's host.
• Log Correlation: Reviewing agent logs for crashes, initialization errors, or timeout patterns immediately preceding probe failures.
• Canary Analysis: Comparing health rates between baseline and canary deployments to isolate issues to a specific agent version.

For autonomous agents, health checks must validate unique runtime states beyond typical web services.

• Context Window Saturation: A probe could check that the agent's session manager is not permanently stuck in a state with a full, uncleared context window.
• Tool Registry Integrity: Verifying that the agent's internal registry of available tools and their schemas is loaded and parseable.
• Planning Engine Heartbeat: Ensuring the core reasoning or planning loop module is initialized and responsive to a trivial planning request.
• Memory Connection: Confirming connectivity to essential state stores (vector databases, SQL databases for episodic memory). Failure in these areas renders the agent non-functional, even if its HTTP server is technically responding.