Data Service Level Objective (Data SLO)

The Service Level Indicator (SLI) is the foundational, quantitative measurement of a specific aspect of service performance. For data, this is a direct measurement of a data quality dimension.

• Examples: (Freshness SLI) = Percentage of dashboard queries served with data updated within the last 1 hour. (Completeness SLI) = Percentage of non-null values in a critical customer ID field over the last 24 hours.
• Key Property: An SLI is a measurement, not a goal. It answers the question, 'What are we measuring?'

The Objective is the target threshold applied to the SLI, defining the minimum acceptable level of service. It transforms a measurement into a commitment.

• Format: Typically expressed as a percentage over a rolling time window (e.g., '99.9% of the time over a 30-day window').
• Example: 'The Freshness SLI must be ≥ 99.9% over a 30-day rolling window.' This means data can be stale for no more than 43.2 minutes per month.
• Precision: The objective must be realistic, measurable, and business-aligned. An unrealistic target (e.g., 100%) renders the SLO useless.

The Error Budget is the permissible amount of service failure derived directly from the SLO. It is the mathematical complement of the objective.

• Calculation: If the SLO is 99.9% freshness, the error budget is 0.1% of the time window. Over 30 days (43,200 minutes), the budget is 43.2 minutes of stale data allowance.
• Primary Function: It serves as a shared resource for balancing reliability versus innovation. Teams can 'spend' the budget on risky deployments or data pipeline changes. Exhausting the budget triggers a blameless post-mortem and a focus on stability.
• Philosophy: An error budget is a tool for managed risk, not a punishment for failure.

Every effective Data SLO is explicitly tied to a specific consumer and a critical use case. This defines the 'service' in Service Level Objective.

• Consumer Identification: Is the data product serving a real-time ML feature store, a executive financial dashboard, or a batch analytics platform? Each has vastly different reliability requirements.
• Use Case Definition: The SLO must protect a business outcome. Example: 'This SLO ensures the Monthly Revenue Report for the CFO is accurate and on-time.'
• Impact: Without this component, SLOs become arbitrary technical metrics disconnected from business value. It answers, 'Who relies on this, and for what?'

The measurement window (e.g., 30-day rolling) and alerting policy define the operational cadence for evaluating the SLO and triggering human intervention.

• Short vs. Long Windows: A burn rate alert monitors the rapid consumption of the error budget over a short window (e.g., '30% of monthly budget burned in 6 hours'), enabling urgent response. The primary SLO is evaluated over a long window for strategic assessment.
• Alerting Tiers: Warning Alerts fire when budget burn is elevated. Critical Alerts fire when budget exhaustion is imminent. This prevents alert fatigue.
• Goal: Alerts should signal actionable risk to the error budget, not every single SLI dip, which may be noise.

A Data SLO is only as good as its automated implementation. This involves instrumenting pipelines to compute SLIs, track budgets, and integrate with incident management.

• Core Systems: Requires a data observability platform or custom instrumentation to compute metrics like freshness, completeness, and accuracy in real-time.
• Automation Checks: Data quality gates in pipelines can block promotions or trigger rollbacks if a change would violate SLOs.
• Integration: SLO status and error budget burn must be visible in dashboards (e.g., Grafana) and ticketing systems (e.g., PagerDuty, Jira).
• Outcome: This component ensures the SLO is a live, governing contract, not a static document.

A Data Service Level Indicator (Data SLI) is the direct, quantitative measurement of a specific aspect of a data product's service level. It is the raw metric used to evaluate compliance with a Data SLO.

• Examples: "Percentage of records arriving within 5 minutes of source event" (freshness), "Percentage of non-null values in critical customer ID field" (completeness), "Percentage of records passing validity rules" (validity).
• Relationship to SLO: An SLO is a target value for an SLI (e.g., SLO: "Customer ID completeness SLI must be ≥ 99.9% over 30 days").

A Data Error Budget is the explicit, allowable amount of time that a data product or pipeline can fail to meet its Data SLOs before triggering a formal incident or mandatory remediation effort. It operationalizes SLOs by defining the cost of unreliability.

• Calculation: Typically derived from the SLO. If an SLO is 99.9% freshness over a 30-day month, the error budget is 0.1% of that time: 43.2 minutes.
• Purpose: Guides engineering priorities. Spending the budget on planned changes (e.g., schema migrations) is acceptable; unplanned consumption from incidents requires investigation and prevention work.

A Data Quality Dimension is a fundamental category or aspect used to characterize, measure, and manage the fitness of data. Dimensions define the what that Data SLIs measure and Data SLOs target.

• Core Dimensions: These include Accuracy, Completeness, Consistency, Validity, Uniqueness, Timeliness, and Freshness.
• Application: A Data SLO is always defined in the context of one or more dimensions. For instance, a "Completeness SLO" or a "Freshness SLO."

A Data Quality Gate is an automated checkpoint within a data pipeline that evaluates one or more data quality metrics (SLIs) against predefined thresholds (SLOs) and can halt processing, trigger alerts, or route data for quarantine if violations are detected.

• Function: Enforces SLOs programmatically at the point of ingestion or transformation.
• Example: A gate after a raw data load that checks for an unexpected 50% drop in row count (completeness SLI) and fails the job before it propagates bad data downstream.

Data Downtime is a reliability metric that quantifies the total period during which a dataset or data product is inaccurate, missing, stale, or otherwise unfit for its intended use. It is the realized consequence of consuming a Data Error Budget.

• Measurement: Often expressed in absolute time (e.g., "12 hours of downtime this quarter") or as a percentage of total available time.
• Relationship to SLO: Tracking downtime is a direct way to measure SLO compliance. If an SLO is violated, data is considered "down" for that period.

A Data Health Index is a high-level, composite metric that synthesizes various technical Data Quality Scores and operational statuses (including SLO adherence) to provide a single, business-facing indicator of the overall fitness-for-use of a data asset, pipeline, or domain.

• Purpose: Communicates overall data status to non-technical stakeholders. An index might aggregate scores for freshness, completeness, and accuracy SLOs.
• Contrast with SLO: While an SLO is a binary target for a specific metric, a Health Index is a graded, holistic score often used for dashboards and executive reporting.