Service Level Objective (SLO)

An SLO must be a quantifiable metric derived from a Service Level Indicator (SLI). It is not a subjective goal but a precise, numerical target against which performance is continuously measured.

• Examples for TinyML: Inference latency (< 100 ms), model accuracy (≥ 95%), system uptime (99.9%), or power consumption per inference (< 10 mJ).
• Key Property: The metric must be directly observable from system telemetry, such as logs or performance counters on the device.

SLOs are always defined over a specific time window, which determines the measurement and evaluation period for compliance. This window aligns the technical target with business or operational review cycles.

• Common Windows: 28-day or 30-day rolling windows are standard for assessing long-term reliability.
• TinyML Context: For real-time sensor processing, a shorter window (e.g., 1 hour) might be used to catch rapid performance degradation. The SLO is typically expressed as a proportion of good events within that window, e.g., "99% of inferences complete within 100 ms over a 30-day window."

The error budget is a fundamental concept derived from an SLO. It is the allowable amount of unreliability, calculated as 1 - SLO. This budget creates a shared, quantified resource for balancing innovation (releases, updates) with stability.

• Calculation: A 99.9% uptime SLO over 30 days permits an error budget of 0.1%, or approximately 43 minutes of downtime.
• Operational Use: In TinyML MLOps, this budget is consumed by model rollouts that may cause instability, failed Over-the-Air (OTA) updates, or periods of model drift. Exhausting the budget should trigger a freeze on risky changes.

Effective SLOs are defined from the end-user or consumer's perspective, measuring aspects of the service that directly impact their experience. Internal system metrics are only useful if they correlate with this external view.

• TinyML Example: For a wake-word detection model, the key user-facing metric is the percentage of correctly recognized commands, not just the CPU utilization of the microcontroller. Latency SLOs should reflect the total time from sensor input to actionable output, not just the neural network inference time.

An SLO must be actionable. Breaching or nearing the exhaustion of the error budget should trigger well-defined operational procedures, not just passive monitoring. This is often implemented through alerting systems.

• Best Practice: Alert on burn rate—how quickly the error budget is being consumed—rather than on every single SLO violation. This prevents alert fatigue.
• TinyML Integration: Alerts can be configured in device management platforms to trigger when fleet-wide metrics, like aggregate inference latency, trend toward the SLO threshold, prompting investigation into potential canary deployment failures or hardware issues.

An SLO sits between the Service Level Indicator (SLI) and the Service Level Agreement (SLA) in the hierarchy of service management.

• SLI (The Measurement): The raw measurement of a service aspect (e.g., p95 latency is 85ms).
• SLO (The Internal Target): The goal for that SLI (e.g., p95 latency < 100ms). It is an internal engineering target.
• SLA (The External Contract): A formal agreement with consequences (like financial penalties) if SLOs are not met. SLOs are typically set more aggressively than SLAs to provide a safety buffer.

A Service Level Indicator (SLI) is the raw, measured metric upon which an SLO is based. It is a quantitative measure of a specific aspect of a service's performance or reliability.

• Examples: Request latency (p99), error rate (successful requests / total requests), throughput (requests per second), availability (uptime percentage).
• Role: An SLI provides the factual data. An SLO defines the target range for that data. For a TinyML device, a critical SLI might be inference latency measured in milliseconds.

A Service Level Agreement (SLA) is a formal, often contractual, commitment between a service provider and a customer that includes consequences (like financial penalties) if SLOs are not met.

• Relationship to SLOs: An SLO is an internal target. An SLA is the external promise. Teams typically set SLOs more aggressively than the SLA to create a buffer and avoid breaching the contract.
• TinyML Context: For a fleet of microcontroller-based sensors, an SLA might guarantee 99.9% data delivery uptime, with SLOs internally set at 99.95% to ensure compliance.

An Error Budget is the calculated amount of unreliability a service can tolerate within a defined period (e.g., a month) before violating its SLO. It quantifies the trade-off between reliability and innovation.

• Calculation: If your SLO is 99.9% availability per month, your error budget is 0.1% of that time, or 43.2 minutes of allowed downtime.
• Usage: Teams can "spend" this budget on deploying risky changes or new features. Exhausting the budget triggers a focus on stability over new releases. For device fleets, this budgets for failed OTA updates or inference errors.

A Canary Deployment is a release strategy where a new software version or model is initially deployed to a small, representative subset of devices or users to monitor its performance and stability before a full rollout.

• Purpose: To validate changes in a low-risk, production-like environment. It directly supports SLO validation by testing if the new version degrades key SLIs.
• TinyML Example: Deploying a new, quantized neural network to 5% of a microcontroller fleet to monitor its inference latency and power consumption against SLOs before updating the entire fleet.

Model Monitoring is the practice of continuously tracking a deployed machine learning model's performance, data quality, and operational health to detect issues like drift, degradation, or infrastructure failures.

• Direct SLO Link: Model monitoring systems are the primary source for SLIs related to model performance, such as prediction accuracy, data drift scores, and inference latency.
• TinyML Specifics: On-device monitoring might track peak memory usage, CPU utilization, and input data distribution to ensure the model operates within its constrained SLOs.

An Over-the-Air (OTA) Update is a method of wirelessly distributing new software, firmware, or machine learning models to a fleet of remote devices without requiring physical access.

• SLO Criticality: The reliability and safety of the OTA update process itself is governed by SLOs. Key metrics include update success rate, rollback success rate, and update duration.
• Risk Mitigation: OTA systems enable canary deployments and rollbacks, which are essential mechanisms for maintaining SLOs during model or firmware updates across thousands of constrained devices.