Service Level Indicator (SLI)

An SLI must be a quantifiable metric derived from observable system data, not a subjective opinion. It is calculated from raw telemetry like request counts, error logs, or latency measurements. Examples include:

• Request latency: The time taken to successfully process a request (e.g., 95th percentile latency < 200ms).
• Error rate: The proportion of requests that result in a failure (e.g., (failed requests / total requests) * 100).
• Throughput: The number of requests a system handles per second.
• Availability: The proportion of time a service is operational and responding.

Effective SLIs measure aspects of the service that end-users directly perceive as quality. They should answer the question: "What does a good experience look like for our users?"

• User-facing latency is a better SLI than internal CPU utilization.
• HTTP 5xx error rate is more relevant than low-level disk I/O errors, unless those errors cause user-visible failures.
• The definition of a 'successful' request must align with the user's goal (e.g., a search returning relevant results, not just a 200 OK).

An SLI is not a single data point but a statistical aggregation over a defined time window and request population. This prevents noise from triggering unnecessary alerts.

• Time Window: SLIs are evaluated over periods like 1 minute, 5 minutes, or 28 days (rolling).
• Aggregation Method: Common methods include:
  • Ratio: (Good events / Total eligible events) over the window.
  • Distribution: Percentiles (p50, p95, p99) of a measurement like latency.
  • Threshold: Percentage of time a metric is below/above a target.
• Example: "The proportion of HTTP requests that succeeded over the last 5 minutes."

An SLI is meaningless without a target threshold defined in an SLO. The SLO sets the acceptable performance level for the SLI.

• SLI: The measurement itself (e.g., error rate calculated as 0.5%).
• SLO: The target for that measurement (e.g., error rate ≤ 0.1%).
• The error budget is then derived from this pairing: it's the allowable deviation from the SLO (e.g., 0.4% of requests can fail before the budget is exhausted). This creates a clear, data-driven framework for deciding when to halt deployments or prioritize reliability work.

SLIs must be computed from high-fidelity, production-grade observability data. The measurement system must be more reliable than the service it monitors.

• Data Sources: Application logs, structured metrics from exporters (Prometheus), distributed traces (OpenTelemetry), or load balancer access logs.
• Instrumentation Points: SLIs should be measured as close to the user as possible, often at the service entry point (e.g., API gateway, load balancer).
• Avoiding Bias: The measurement must cover all relevant traffic. Sampling can introduce bias and invalidate SLI calculations for low-volume services.

For AI-powered services, SLIs must capture both infrastructure health and model quality. Key examples include:

• Inference Latency: p95 latency for model prediction requests.
• Model Throughput: Predictions per second the endpoint can handle.
• Inference Error Rate: Percentage of prediction requests returning a 5xx error or a system-level failure.
• Model Quality Drift: Percentage of predictions where confidence scores diverge significantly from a baseline, indicating potential performance degradation.
• Hallucination Rate (for LLMs): Proportion of generated outputs flagged as factually incorrect or unsupported by the provided context.
• Data Freshness: Age of the most recent data used for a prediction in a real-time system.

An error budget is the calculated amount of allowable unreliability for a service, derived from its Service Level Objective (SLO). It is defined as 1 - SLO.

• Calculation: If your SLO is 99.9% availability, your error budget is 0.1% unreliability over the compliance period.
• Function: The error budget quantifies how much risk a team can take. Introducing new features, performing deployments, or conducting experiments consumes this budget.
• Canary Analysis Link: A failed canary that causes a significant increase in errors consumes the error budget, triggering an automated rollback to preserve reliability.

A Service Level Agreement (SLA) is a formal contract between a service provider and a customer that includes Service Level Objectives (SLOs) and specifies the consequences (e.g., financial penalties) for failing to meet them.

• Key Difference from SLO: An SLO is an internal engineering goal. An SLA is an external, contractual commitment.
• Hierarchy: SLIs are measured to determine if SLOs are met. SLOs are set more stringently than SLAs to provide a safety buffer.
• Example: An internal SLO might be 99.95% uptime to comfortably guarantee a customer-facing SLA of 99.9%.

Golden Signals are four high-level metrics that provide a comprehensive view of a service's health from a user's perspective. They are the primary candidates for defining Service Level Indicators (SLIs).

• Latency: The time it takes to service a request. SLI example: 95th percentile request duration.
• Traffic: The demand placed on the system. SLI example: HTTP requests per second.
• Errors: The rate of failed requests. SLI example: percentage of HTTP 5xx responses.
• Saturation: How "full" the service is. SLI example: memory utilization percentage or queue depth.

These signals are foundational for canary analysis, where they are compared between the baseline and new deployment.

Synthetic monitoring involves using scripted, simulated transactions to proactively test and measure the performance and availability of a service from external points. It is a key source of data for Service Level Indicators (SLIs).

• Purpose: To measure service health and SLI compliance from a user's geographic perspective before real users are affected.
• Contrast with RUM: Unlike Real User Monitoring (RUM), which measures actual user traffic, synthetic monitoring uses controlled, predictable scripts.
• Canary Use Case: Synthetic probes can be directed at canary instances to validate functionality and performance (latency, success rate) as part of the deployment validation process, providing early warning signals.