Service Level Agreement (SLA)

An SLO is a quantitative target for service reliability or quality, expressed as a percentage over a time window. For AI services, SLOs extend beyond traditional uptime to include:

• Model Quality SLOs: e.g., "99% of responses must have a hallucination rate below 5%."
• Performance SLOs: e.g., "p95 inference latency must be under 500ms."
• Business SLOs: e.g., "Agent task success rate must exceed 95%." SLOs are derived from Service Level Indicators (SLIs) and define the threshold for acceptable service.

An SLI is a directly measurable metric quantifying a specific aspect of service performance. AI-specific SLIs are critical for evaluating SLOs:

• Inference Latency: Time from request to model output (often split into Time To First Token (TTFT) and Time Per Output Token (TPOT)).
• Error Rate: Percentage of requests resulting in a model error or crash.
• Quality Metrics: Hallucination rate, answer faithfulness score, or Retrieval Precision@K for RAG systems.
• Throughput: Queries processed per second (QPS) with techniques like continuous batching.

An error budget is the allowable amount of unreliability, calculated as 100% - SLO. It quantifies risk tolerance.

• Purpose: Defines how many errors or SLO violations are acceptable before financial or operational penalties apply.
• Burn Rate: The speed at which the error budget is consumed. A high burn rate triggers alerts.
• Remediation Actions: Specifies steps if the budget is exhausted, which may include service credits, automatic rollbacks via canary deployment analysis, or dedicated engineering time for remediation.

AI SLAs require clauses addressing the probabilistic nature of models:

• Hallucination & Factual Accuracy: Defines acceptable rates of unsupported output and methods for detection.
• Data Drift & Model Decay: Specifies monitoring for data drift detection and obligations for model retraining.
• Output Consistency: May include targets for variance in responses to identical inputs.
• Context Window & Memory Limits: Defines limits on input token length and agentic context management.
• Tool Calling Reliability: For agentic systems, specifies success rates for external API execution.

This section defines how compliance is measured and reported, and what is excluded from SLA calculations.

• Measurement Methodology: How SLIs are collected (e.g., client-side telemetry, server logs) and aggregated (e.g., percentile latency p95, p99).
• Reporting Frequency: Regular SLA compliance reports (e.g., monthly).
• Exclusions (Force Majeure): Standard exclusions for outages beyond provider control, plus AI-specific exclusions such as:
  • Attacks via adversarial testing or prompt injection.
  • Use outside defined Critical User Journeys (CUJs).
  • Use of unsupported input data formats or volumes.

The contractual framework that operationalizes the technical metrics.

• Service Credits: Financial penalties or credits applied if SLOs are not met, often tiered based on error budget consumption.
• Termination Rights: Conditions under which the customer can terminate the agreement for chronic SLA failures.
• Data Ownership & Security: Asserts customer ownership of input data and output, and defines security standards.
• Governance & Audit: Rights for customers to audit SLA measurements and the provider's AI governance practices.
• Liability Caps: Limits on total liability, which is crucial given the potential impact of AI errors.

Defines the maximum allowable time for a model to return a prediction and the minimum number of requests it must handle per second.

• Time to First Token (TTFT): Latency from request start to first output token for LLMs.
• Time Per Output Token (TPOT): Latency for each subsequent token in a stream.
• Throughput (QPS): Queries per second, often with continuous batching to optimize GPU use.
• Tail Latency (p95, p99): Guarantees for the slowest requests, which most impact user experience.

Example: "p95 end-to-end inference latency shall not exceed 250ms for 99.9% of requests over any 5-minute window."

Guarantees the statistical performance of the model against a held-out evaluation dataset or live traffic.

• Accuracy/Precision/Recall/F1 Score: Minimum thresholds for classification tasks.
• Hallucination Rate: Maximum permissible percentage of factually incorrect or unsupported outputs for generative models.
• Answer Faithfulness/Attribution: For RAG systems, the degree to which answers are grounded in provided source documents.
• Instruction Following Accuracy: Measures adherence to complex prompt constraints.

Example: "The model shall maintain a hallucination rate below 2% as measured by automated fact-checking against the provided context over a monthly period."

Specific to Retrieval-Augmented Generation architectures, these clauses govern the quality of the document retrieval step.

• Retrieval Precision/Recall@K: Proportion of relevant documents in the top K results.
• Retrieval Latency: Time to fetch and rank context documents.
• Context Relevance Score: Semantic match between query and retrieved chunks.
• Source Attribution Completeness: Requirement to cite specific document passages.

Example: "The retrieval system shall achieve a Precision@5 of ≥85% for all queries, as evaluated by a monthly human audit sample."

For autonomous AI agents, this defines the reliability of multi-step task completion.

• End-to-End Task Success Rate: Percentage of user-initiated tasks completed without human intervention.
• Tool Calling Success Rate: Reliability of external API executions.
• Reasoning Trace Validity: Logical correctness of the agent's step-by-step reasoning.
• Mean Time To Recovery (MTTR): For agent failures, the average time to auto-recover or escalate.

Example: "The customer service agent shall successfully resolve 92% of tier-1 support tickets within the defined workflow without escalation over a quarterly period."

Clauses ensuring data privacy, model stability, and compliance with regulations.

• Data Drift/Concept Drift Detection: Commitment to monitor and alert on significant input distribution shifts.
• Model Version Rollback SLA: Maximum time to revert to a previous stable model version.
• Privacy & Security: Adherence to differential privacy guarantees or data encryption standards.
• Bias & Fairness Audits: Schedule and methodology for evaluating model performance across protected classes.

Example: "Provider will perform weekly statistical tests for data drift and notify Customer within 2 hours if drift exceeds a PSI threshold of 0.2."

Ties AI service reliability to business outcomes and defines remedies for failure.

• Error Budget Consumption (Burn Rate): Defines alerting thresholds based on the rate of SLO violation.
• Composite SLO: Overall reliability score derived from multiple AI-specific SLIs.
• Business Metric Correlation: SLA may be linked to downstream metrics like conversion rate or customer satisfaction (CSAT).
• Service Credits/Penalties: Financial remedies defined per violation, often scaling with severity and duration.

Example: "If the composite SLO for the recommendation service falls below 99.5% for a calendar month, Customer will receive a service credit equal to 15% of that month's fees."

A Service Level Objective (SLO) is the quantitative target that forms the core of an SLA. It defines the acceptable level of service reliability, typically expressed as a percentage over a rolling time window (e.g., "99.9% of requests must have latency < 200ms this month"). SLOs are internal goals that, when met, ensure customer satisfaction and prevent SLA breaches.

• Purpose: To create a clear, measurable target for engineering teams.
• Example: "The p95 latency for text completions must be ≤ 500ms for 99% of requests in a 30-day window."

A Service Level Indicator (SLI) is the raw, measurable metric that quantifies an aspect of service performance. It is the direct measurement used to evaluate an SLO. For AI services, key SLIs include:

• Model Inference Latency: End-to-end time from request to final response.
• Error Rate: Percentage of requests resulting in a 5xx HTTP error or a model-serving failure.
• Throughput: Queries processed per second (QPS).
• Quality Metrics: Such as hallucination rate or retrieval precision. An SLI becomes meaningful when paired with an SLO target.

An Error Budget is the explicit, calculated amount of unreliability a service can tolerate without violating its SLO. It is derived as 100% - SLO Target. If the SLO is 99.9% availability, the error budget is 0.1% of unsuccessful requests over the compliance period.

• Function: It quantifies risk and governs release velocity. Teams can deploy new features as long as they don't exhaust the budget.
• Management: Exhausting the budget should trigger a freeze on feature launches and a focus on stability and remediation. It transforms SLOs from abstract goals into a central resource for managing engineering trade-offs.

Golden Signals are four high-level metrics that provide a comprehensive view of any service's health, as defined in Site Reliability Engineering (SRE). They are foundational for defining SLIs:

• Latency: The time to service a request (focus on tail percentiles like p95, p99).
• Traffic: The demand on the system (e.g., queries per second, concurrent users).
• Errors: The rate of failed requests (e.g., HTTP 5xx, model inference failures).
• Saturation: How "full" the service is (e.g., GPU memory utilization, queue depth). Monitoring these signals provides the baseline data needed to create meaningful, user-centric SLOs.

A Critical User Journey (CUJ) is a specific, end-to-end sequence of interactions that is essential to user success. SLOs should be derived from CUJs rather than low-level system metrics to ensure they reflect real user experience.

• For AI Services: A CUJ could be "User submits a complex query, the RAG system retrieves relevant context, and the LLM generates a factual, cited answer."
• Impact: SLIs for this CUJ would measure the end-to-end success rate, latency, and answer faithfulness of this entire flow, ensuring the SLO protects the business value of the service.

Burn Rate measures how quickly a service is consuming its error budget, expressed as a percentage of the budget burned per hour. A burn rate of 100% means the budget will be exhausted in the SLO's time window.

• Multi-Window Alerting is a sophisticated strategy that uses burn rate to trigger alerts. It sets different thresholds for short and long time windows to distinguish between brief incidents and sustained degradation.
  • Example: Alert if the budget burns at 10x rate for 1 hour (fast, short-term alert) OR at 2x rate for 6 hours (slower, long-term alert). This reduces alert fatigue while ensuring timely response to real SLO risks.