Service Level Objective (SLO)

The Service Level Indicator (SLI) is the raw, measurable metric that quantifies a specific aspect of service performance. It is the foundational data point for an SLO. An SLI must be:

• Directly measurable (e.g., request latency, success rate, throughput).
• Well-defined with clear calculation logic (e.g., successful_requests / total_requests).
• Aligned with user experience, measuring what the user actually perceives.

For AI services, common SLIs include model inference latency, error rate (4xx/5xx), and task success rate for autonomous agents.

This is the numerical goal of the SLO, expressed as a threshold the SLI must meet over a defined period. It transforms a metric into a commitment.

• Typically expressed as a percentage or a duration (e.g., 99.9% availability, p95 latency < 200ms).
• The target defines the line between acceptable and unacceptable performance.
• It should be ambitious but achievable, balancing user expectations with engineering reality. A target that is too easy provides no guardrail; one that is too strict leads to constant violation and alert fatigue.

The measurement window is the rolling time period over which the SLI is evaluated against the target. It provides temporal context and stability.

• Common windows are 28 or 30 days, aligning with business cycles.
• Shorter windows (e.g., 1 hour, 1 day) are used for burn rate alerts to catch rapid degradation.
• The choice of window affects SLO sensitivity: a 30-day window smooths out brief incidents, while a 1-day window makes the SLO more reactive to single failures.

The error budget is the permissible amount of unreliability, derived directly from the SLO. It is calculated as 100% - SLO Target.

• If the SLO is 99.9% availability, the error budget is 0.1% of bad time.
• This budget quantifies acceptable risk for innovation. Teams can spend it on deployments, experiments, or accepting known risks.
• Burn rate monitoring tracks how quickly this budget is being consumed, triggering alerts based on the risk of exhausting it before the measurement window ends.

The Critical User Journey (CUJ) is the specific, high-value sequence of user interactions that an SLO is designed to protect. It ensures SLOs are user-centric, not system-centric.

• An SLO should measure the SLI for a complete CUJ, not just an isolated backend API call.
• For an AI chatbot, the CUJ might be "user submits query → system retrieves context → model generates answer → answer is streamed to user."
• Defining the CUJ forces alignment between technical metrics and business outcomes, ensuring the SLO guards what truly matters to the customer.

The alerting policy defines the rules for notifying engineers based on SLO burn rate, not raw metric thresholds. This is a fundamental shift from traditional monitoring.

• Alerts trigger when the error budget burn rate indicates a high probability of exhausting the budget before the measurement window ends.
• Multi-window alerting (e.g., 1-hour and 6-hour burn rates) distinguishes between brief spikes and sustained degradation.
• This approach reduces alert noise and ensures teams are only paged when there is a meaningful risk to the service reliability commitment.

A Service Level Indicator (SLI) is the fundamental, directly measurable metric that quantifies a specific aspect of a service's performance. For AI services, common SLIs include:

• Model Inference Latency: Total time from request to response.
• Time To First Token (TTFT): Latency until the first token of a streaming response.
• Error Rate: Percentage of requests resulting in a failed or invalid response.
• Throughput: Queries processed per second (QPS). An SLO is a target set on one or more of these SLIs over a defined time window.

An Error Budget is the explicit, quantified risk a service team is allowed to take. It is calculated as 100% - SLO. For example, a 99.9% availability SLO over a 30-day window creates a budget of 0.1% unreliability, or approximately 43 minutes of downtime. This budget is consumed by incidents and failed deployments. It serves as a crucial management tool, enabling teams to make data-driven trade-offs between reliability work and feature velocity. Exhausting the budget should trigger a freeze on risky changes.

A Service Level Agreement (SLA) is a formal contract between a service provider and a customer that includes consequences for missing performance targets. It is an external-facing document, often commercial in nature. Key distinctions from an SLO:

• SLOs are internal goals used for engineering management.
• SLAs are external promises with defined remedies (e.g., service credits, financial penalties). An SLA is typically set less aggressively than the internal SLO to provide a buffer, ensuring the SLO is violated before the SLA is breached.

Burn Rate measures how quickly a service consumes its error budget, expressed as a percentage of the total budget consumed per unit of time. A burn rate of 100% means the budget will be exhausted in the current alerting window. Multi-Window Alerting is a critical strategy that uses burn rates across different time windows (e.g., 1-hour and 6-hour) to trigger alerts. This separates urgent, fast-burning fires (short window) from slower, sustained degradations (long window), reducing alert fatigue and focusing response efforts appropriately.

A Critical User Journey (CUJ) is a specific, high-value sequence of user interactions that is essential to the user's success with a service. For an AI chatbot, a CUJ might be: "User submits a query → System retrieves context → Model generates a response → User receives a helpful answer." SLOs should be defined to protect these journeys, not just individual technical components. Monitoring SLIs across an entire CUJ ensures the user-perceived reliability and performance is measured, which is more meaningful than monitoring isolated backend services.

A Composite SLO represents the overall reliability of a complex service that depends on multiple internal components or external dependencies. It is derived mathematically from the SLOs of its constituent parts. For a RAG system, the composite SLO for answering a question might aggregate:

• Retrieval subsystem availability (99.95%)
• Vector database query latency SLO (p95 < 100ms)
• LLM inference success rate (99.9%) The failure of any single component can violate the composite SLO. Calculating composite SLOs is essential for understanding the true reliability of user-facing features built from microservices.