Health Check

A liveness probe determines if a containerized application or agent process is still running and responsive. It is a proactive check for catastrophic failures, such as a deadlocked process. If the probe fails, the orchestrator (e.g., Kubernetes) typically terminates and restarts the container.

• Mechanism: Executes a command, makes an HTTP GET request, or opens a TCP socket against the target.
• Agentic Context: Critical for restarting agents that have entered an unrecoverable state due to logic errors or resource exhaustion.
• Example: An HTTP GET to /health/live returning a 200 status code.

A readiness probe assesses if an application instance has completed its initialization and is prepared to accept network requests. It is distinct from liveness; a failing readiness probe does not trigger a restart but instead removes the pod from service load balancers.

• Purpose: Prevents traffic from being sent to pods that are booting up, loading large models, or connecting to downstream dependencies.
• Agentic Context: Essential for agents with long startup times, such as those loading multi-gigabyte language models or connecting to vector databases. Ensures the agent's tool-calling API is fully operational before receiving user queries.

Health checks are not binary switches but highly tunable mechanisms. Key parameters define their behavior and sensitivity to transient issues:

• initialDelaySeconds: Waits before starting probes after container start.
• periodSeconds: How often to perform the probe.
• timeoutSeconds: Time after which a probe attempt is considered failed.
• successThreshold: Consecutive successes required to mark a failed container as healthy.
• failureThreshold: Consecutive failures required to mark a healthy container as unhealthy.

Proper tuning prevents unnecessary restarts during legitimate, temporary load spikes or garbage collection pauses in an agent's runtime.

Health status is the primary signal used by modern deployment orchestrators to manage rollouts and ensure stability.

• Rolling Updates: The orchestrator waits for new pods to pass their readiness probe before terminating old ones and scaling up the new replica set.
• Canary Deployments: Traffic is only shifted to the new canary pod after it reports as healthy via its readiness probe. A failing liveness probe on the canary triggers an automatic rollback.
• Autoscaling: While typically driven by CPU/memory, custom metrics from health check endpoints can inform scaling decisions (e.g., scale up if average agent response latency from the health endpoint exceeds a threshold).

While HTTP GET is common, orchestrators support multiple probe types for different application architectures:

• Exec Probe: Executes a specified command inside the container. Exits with code 0 for success, any other code for failure. Used for deep, application-specific logic checks (e.g., agentctl validate-state).
• TCP Socket Probe: Attempts to open a TCP connection to a specified port on the container. Success is simply establishing a connection. Ideal for non-HTTP services like gRPC or custom binary protocols used in multi-agent communication.
• gRPC Health Checking Protocol: A standardized health check protocol for gRPC services, natively supported by Kubernetes, providing a more efficient and typed alternative to HTTP for microservices and agent meshes.

For autonomous agents, basic process checks are insufficient. Effective health checks validate the agent's functional capabilities.

A comprehensive agent health endpoint should check:

• Core Process: Is the agent runtime (e.g., Python interpreter, Node.js) alive?
• Model Accessibility: Can the agent load and perform a trivial inference with its primary language model?
• Tool Connectivity: Can the agent establish connections to its critical external dependencies (APIs, databases, vector stores)?
• Memory/Context: Is the agent's session memory or context window within operational limits?
• Orchestrator Heartbeat: For multi-agent systems, can the agent communicate with its central orchestrator or peer agents?

This moves health checking from 'is it running?' to 'is it ready to perform its designed function?'

For autonomous agents, health checks must verify both infrastructure and cognitive state. A comprehensive endpoint should report:

• Infrastructure Health: Container status, memory/CPU usage, and connectivity to dependent services (vector DB, LLM API).
• Agent State: Availability of core components like the planning module, context window status, and tool registry.
• Operational Readiness: Current load, queue depth for pending tasks, and licensing/authentication validity. Example response: {"status": "healthy", "agent_state": "idle", "llm_latency_p99": 450, "tools_available": 12}

This advanced health check executes a canonical, non-destructive workflow to validate end-to-end agent functionality. Instead of a simple ping, it runs a synthetic transaction—a predefined test that mimics a real user request. For a customer service agent, this might involve:

• Parsing a test query.
• Executing a retrieval-augmented generation (RAG) lookup.
• Formulating a response.
• Logging the full execution trace, latency, and correctness of the result. This validates the entire pipeline, from input parsing to tool execution, beyond basic connectivity.

Health checks should feed into Service Level Indicators (SLIs) for autonomous systems. Beyond 'up/down', define SLIs that reflect user experience:

• Planning Success Rate: Percentage of agent sessions where a valid plan is generated.
• Tool Call Error Rate: Proportion of external API executions that fail.
• End-to-End Latency P99: 99th percentile latency for completing a full agent task. Configure health checks to emit these metrics, allowing Service Level Objectives (SLOs) like '99.9% planning success rate over 30 days'. This shifts monitoring from infrastructure health to business-level agent reliability.

A robust health check strategy differentiates between critical and non-critical failures. Implement a tiered status system:

• Healthy: All core dependencies (LLM, main database) are reachable.
• Degraded: A non-critical dependency (e.g., a secondary analytics API) is unavailable, but core agent functions remain operational.
• Unhealthy: A critical dependency failure prevents the agent from performing its primary function. The health endpoint should clearly report the status and list unhealthy dependencies. This allows load balancers to drain traffic from 'unhealthy' instances while still utilizing 'degraded' ones, improving overall system resilience.

A deployment strategy where a new version of an agent is released to a small, controlled subset of production traffic. Health checks and agent performance benchmarking (latency, success rate) on the canary group are compared against the stable version to validate the release before a full rollout.

• Risk Mitigation: Limits the impact of a defective new agent version.
• Data-Driven Rollout: The decision to proceed, pause, or roll back is based on real-time observability data from the canary group.

A resilience pattern that programmatically fails fast when a downstream dependency (e.g., a tool API, LLM endpoint, or database) is unhealthy or unresponsive. It prevents cascading failures and allows the dependent system time to recover.

• Three States: Closed (normal operation), Open (requests fail immediately), Half-Open (testing if dependency has recovered).
• Agent-Specific Use: Protects an agent's tool-calling logic from being blocked by a failing external service, allowing it to potentially use fallback tools or reasoning paths.

The process by which an agent instance completes its in-flight tasks and releases resources (e.g., context sessions, API connections) properly before termination. This is often initiated by the orchestrator sending a SIGTERM signal after a health check indicates a need for replacement.

• Prevents Data Loss: Allows an agent to finish a multi-step plan or save its episodic memory state before shutting down.
• Lifecycle Hook: Implemented using PreStop hooks in Kubernetes, which run before the container is sent the termination signal.

A dedicated infrastructure layer that handles service-to-service communication, providing a unified plane for observability, security, and traffic control. For multi-agent systems, a service mesh can manage health checks, load balancing, and distributed tracing between agent nodes.

• Advanced Health Checks: Provides more sophisticated health assessment than basic HTTP/TCP checks, including protocol-specific validation.
• Traffic Management: Enables fine-grained traffic splitting for canary deployments and automatic failure handling based on health status.