Composite SLO

A composite SLO is not a single metric but a mathematical combination of multiple component SLOs or SLIs. It provides a unified view of service health by calculating an overall reliability percentage from its parts.

• Example: A user-facing API with a 99.9% SLO might depend on a database (99.95% SLO) and a machine learning inference service (99.8% SLO). The composite SLO for the entire API journey is a function of these dependencies.
• Calculation: Often derived using probability (e.g., multiplying availability of serial dependencies) or weighted averages for parallel services.

The primary purpose of a composite SLO is to explicitly model and account for the reliability chain in a distributed system. It forces engineering teams to understand how failures in downstream dependencies propagate to user experience.

• Critical Path Identification: Highlights which underlying services have the greatest impact on the composite target.
• Dependency Graph: The composite SLO is defined by the architecture's dependency graph, whether services are in series (failures multiply) or parallel (failures may be masked).

A composite SLO creates a single, shared error budget for the entire service. This budget is consumed by failures in any underlying component, forcing teams to collaborate on reliability investments.

• Unified Risk Currency: The error budget becomes a common resource. A database team consuming budget through downtime directly impacts the frontend team's ability to ship features.
• Prioritization: Drives data-driven decisions on where to invest engineering effort to protect the overall budget, such as improving the weakest link in the dependency chain.

Composite SLOs are particularly critical for AI-powered services due to their inherently composite nature. A single inference request may trigger a chain of dependent services.

• Typical AI Service Chain: User Request → API Gateway → Feature Store → Model Inference (with potential retries) → Post-Processing → Response.
• Key Component SLIs: Model latency (p99), feature retrieval success rate, inference error rate, and hallucination rate (for generative AI) must all be combined into a user-centric composite SLO.

A valid composite SLO depends on rigorously defined component Service Level Indicators (SLIs). Each SLI must be a measurable, unambiguous metric.

• Good SLI: "The proportion of HTTP requests to /predict that return a successful (2xx) response within 500ms."
• Poor SLI: "Model is fast."
• Consistent Measurement: All component SLIs must be measured over the same time window (e.g., 30 days) and with the same aggregation logic to be combined meaningfully.

By making dependency costs visible, composite SLOs directly influence system design. Teams are incentivized to architect for reliability.

• Decoupling: May motivate adding caching layers or circuit breakers to isolate failures of a low-SLO dependency.
• Graceful Degradation: Encourages defining fallback behaviors (e.g., serving a slightly stale model) to protect the composite SLO when a component is degraded.
• Dependency Selection: Provides a framework for evaluating third-party services based on their contribution to the overall SLO.

A Retrieval-Augmented Generation (RAG) system's composite SLO aggregates the performance of its constituent services. The overall SLO for a correct, timely answer is a function of:

• Retrieval Latency SLI: Time to fetch relevant context from a vector database.
• Retrieval Precision@K SLI: Relevance of the top documents retrieved.
• Generation Latency SLI: Time for the LLM to produce a final answer (TTFT + TPOT).
• Answer Faithfulness SLI: Percentage of claims in the answer supported by retrieved context. A violation in any component (e.g., slow retrieval or high hallucination rate) can consume the composite error budget, requiring holistic optimization.

For a multi-step AI agent, the composite SLO for task success rate depends on the reliability of each step in its cognitive loop. This SLO is a logical AND of component SLOs:

• Tool Calling Success Rate: Percentage of API calls that execute without error.
• Reasoning Validity: Correctness of the agent's step-by-step plan, evaluated via trace evaluation.
• Context Window Management: Ability to maintain relevant conversation history without exceeding limits.
• Error Correction Loop Success: Rate at which the agent can self-correct from failures using recursive workflows. The overall agent SLO is only satisfied if all critical sub-tasks meet their individual objectives, making it highly sensitive to the weakest link.

A vision-language-action model for robotics has a composite SLO for perception-to-action latency. This objective combines latencies across heterogeneous subsystems:

• Frame Capture & Preprocessing SLI: Latency of image sensor data ready for inference.
• Visual Encoding SLI: Time for a vision transformer to process an image frame.
• Cross-Modal Fusion SLI: Latency to align visual features with a natural language command.
• Action Policy Generation SLI: Time to compute actuator commands (e.g., joint angles). The end-to-end p99 latency SLO must account for the tail latency amplification that occurs as variances in each stage compound, requiring careful pipeline design and parallelization.

A composite SLO can directly tie technical metrics to a key business outcome. For an AI-driven recommendation service, the top-level SLO might be 'Recommendation Click-Through Rate (CTR) > Target'. This composite objective is derived from:

• Model Inference Latency SLO: Slow predictions degrade user engagement.
• Personalization Relevance SLO: Measured via offline NDCG or online A/B test metrics.
• Model Freshness SLO: Maximum allowable age of training data to prevent concept drift.
• Feature Store Availability SLO: Uptime for the service providing real-time user embeddings. The business metric correlation is modeled, and the technical SLOs are set to protect the composite business objective, aligning engineering and product goals.

In cost-sensitive deployments, a composite SLO balances performance, quality, and expenditure. For a high-volume query service using a large language model, the SLO might be: 'Achieve p95 latency < 2s AND cost per query < $0.001 with 99.9% availability.' This requires aggregating:

• Latency SLOs for TTFT and TPOT.
• Model Quality SLOs like instruction-following accuracy, which may drop with aggressive optimization.
• Infrastructure Cost SLIs: GPU utilization, dynamic batching efficiency, and cache hit rates.
• Availability SLO for the autoscaling inference endpoint. Teams manage an error budget for both latency and cost, making trade-off decisions explicit when deploying new model versions or optimization techniques like quantization.

For a privacy-preserving healthcare federated learning system, a composite SLO governs the timely completion of a training round across distributed edge nodes. This SLO is a function of:

• Participant Availability SLI: Percentage of client devices (e.g., hospitals) that are online and available for training.
• Round Communication Latency SLI: Time to aggregate model updates from all participants.
• Update Quality SLI: Validation that participant updates are not malicious (e.g., via anomaly detection).
• Global Model Convergence SLI: Improvement in a centralized validation metric after each round. The composite SLO defines the acceptable time and participation rate to achieve a target model improvement, accounting for the inherent unpredictability of edge device networks and schedules.

A Service Level Objective (SLO) is a quantitative target for the reliability, performance, or quality of a service, expressed as a percentage of requests that must meet a specific Service Level Indicator (SLI) over a defined time window. It is the fundamental building block from which a Composite SLO is constructed.

• Example: "99.9% of API requests must have a latency under 200ms over a 30-day rolling window."
• SLOs define the acceptable level of service unreliability, creating an Error Budget for engineering teams.

A Service Level Indicator (SLI) is a directly measurable metric that quantifies a specific aspect of a service's performance, such as latency, error rate, or throughput. SLIs serve as the raw data inputs for evaluating Service Level Objectives (SLOs).

• Examples: Request latency (p95), HTTP success rate (5xx errors), system throughput (queries per second).
• A Composite SLO mathematically combines multiple SLIs (e.g., latency and success rate) or SLIs from different service components into a single, holistic reliability target.

An Error Budget is the allowable amount of service unreliability, calculated as 100% - SLO. It quantifies the risk a team can accept for deploying changes or experiencing failures without violating the SLO.

• Example: A 99.9% SLO leaves a 0.1% error budget.
• For a Composite SLO, the error budget is derived from the combined performance of all underlying components. The burn rate of this composite budget is a critical alerting signal, indicating whether the entire system is at risk of breaching its overall reliability target.

A Critical User Journey (CUJ) is a specific, high-value sequence of user interactions essential to user success. Composite SLOs are often defined to protect these journeys, which typically span multiple backend services and dependencies.

• Example: "User login -> product search -> add to cart -> checkout."
• The SLO for the entire checkout CUJ would be a Composite SLO, aggregating the SLIs (latency, error rate) of the authentication, catalog, cart, and payment services. This ensures the user-perceived reliability of the complete workflow is measured and guaranteed.

Burn Rate is the speed at which a service consumes its error budget, expressed as the percentage of the budget consumed per unit of time (e.g., % per hour). It is a key metric for triggering high-fidelity alerts.

• Multi-window alerting uses burn rates over short (e.g., 1h) and long (e.g., 30d) windows to distinguish brief spikes from sustained degradation.
• For a Composite SLO, the burn rate must be calculated based on the aggregated performance of all components. A fast burn rate on the composite SLO signals that the overall system is deteriorating, even if individual component SLOs are still within budget.

Tail Latency Amplification is a phenomenon in distributed systems where the slowest percentile of requests (e.g., p99) becomes significantly slower due to serial dependencies, queuing, and resource contention. This is a critical consideration for Composite SLOs.

• In a workflow with 10 sequential services, each with a p99 latency of 100ms, the user-facing p99 latency can amplify to nearly 1 second.
• A Composite SLO for end-to-end latency must account for this amplification effect. Defining the SLO requires modeling or measuring the actual tail latency of the entire dependency chain, not just summing the average latencies of individual components.