Data Correctness SLO

The foundation of any correctness measure is a set of programmatic validation rules that encode domain-specific business logic. These rules define what "correct" means for the data. Common types include:

• Format and type validation (e.g., email addresses must match a regex pattern).
• Range and boundary checks (e.g., transaction_amount must be positive).
• Referential integrity checks (e.g., customer_id must exist in the master customer table).
• Cross-field logical consistency (e.g., ship_date cannot be before order_date). Without explicit, executable rules, correctness cannot be measured objectively.

The Service Level Indicator (SLI) is the precise, quantitative measurement of correctness. It is typically expressed as the proportion of valid data points over a defined window. The standard formula is: SLI = (Total Records - Invalid Records) / Total Records

Key considerations for the SLI:

• Measurement Window: The period over which the SLI is calculated (e.g., hourly, daily).
• Aggregation Level: Whether the SLI is measured per table, column, or data product.
• Invalid Record Definition: Clear logic for what constitutes an "invalid" record based on the validation rules. This raw metric is the input for evaluating SLO compliance.

The SLO target is the specific, agreed-upon threshold for the SLI, defining the minimum acceptable level of correctness. It is expressed as a percentage over a compliance period (e.g., "99.9% of records must be valid each month").

The Error Budget is derived directly from the SLO. If the SLO is 99.9%, the error budget is 0.1%. This represents the allowable "unreliability" or invalid data. This budget is a crucial management tool:

• It quantifies the trade-off between innovation (e.g., launching new data features) and reliability.
• Depleting the error budget should trigger a focus on data quality improvements and a freeze on risky changes.

A correctness SLO requires robust instrumentation to compute the SLI in near real-time. This infrastructure includes:

• In-line or post-process validators: Systems that apply business logic rules to data as it flows or after it lands.
• Telemetry collection: Capturing validation results (pass/fail counts) with precise timestamps.
• Time-series databases & dashboards: Storing SLI history and providing visibility into current and historical compliance.
• Alerting systems: Triggering notifications when SLI trends indicate imminent SLO violation or error budget burn is too high. Without this observability, an SLO is merely a theoretical document.

A Data Correctness SLO must have a clear owner (e.g., a data product manager or engineering team) accountable for its health. An escalation policy defines the actions taken at different thresholds of error budget consumption. For example:

• Warning (50% budget burned): Notify the team for investigation.
• High Alert (80% burned): Escalate to engineering leadership, begin postmortem.
• Budget Exhausted (100%): Initiate a formal remediation plan, potentially halt deployments that affect the related data pipeline. This policy turns metrics into accountable operational processes.

Correctness is not isolated. A value may be correct according to its table's rules but become an error due to a corrupted upstream source. Effective SLOs are informed by data lineage. This allows:

• Impact Analysis: Understanding which downstream SLOs are at risk when an upstream correctness SLI degrades.
• Root Cause Diagnosis: Quickly tracing a correctness failure back to the specific transformation or source that introduced the invalid data.
• Dependency Mapping: Defining SLO hierarchies, where a core source data SLO supports multiple derived product SLOs.

A Service Level Objective (SLO) is a quantitative, internal target that defines the acceptable level of reliability for a specific service metric over a defined period. For data systems, this shifts the focus from binary "up/down" status to measurable quality attributes like correctness, freshness, and completeness.

• Core Concept: The target you aim to achieve (e.g., "99.9% of records must pass validation").
• Foundation for Error Budgets: The SLO is used to calculate the allowable error budget.
• Different from SLA: An SLO is an internal engineering goal, whereas a Service Level Agreement (SLA) is an external contract with customers.

A Service Level Indicator (SLI) is a quantitative measure of a specific aspect of a service's performance, which is used to evaluate compliance with an SLO. For data correctness, an SLI is the actual measurement of invalid data.

• The Measured Metric: The raw measurement (e.g., "Percentage of records failing business logic validation this week").
• Direct Input to SLO: The SLI is compared against the SLO target to determine if the objective is being met.
• Examples for Correctness: Invalid value rate, schema violation rate, referential integrity failure rate.

An Error Budget is the allowable amount of unreliability, calculated as 100% - SLO. It quantifies how much "bad" data or downtime is acceptable, creating a shared resource for balancing innovation velocity with reliability.

• Trade-off Mechanism: Allows teams to consciously spend the budget on risky changes (e.g., schema migrations) or must preserve it by slowing down.
• Governs Priorities: When the error budget is depleted, the focus must shift exclusively to improving reliability.
• Data Context: A Data Correctness Error Budget might be "We can tolerate 0.1% invalid records this quarter."

A Data Freshness SLO defines the maximum acceptable age of data, specifying how long data can be delayed from its source event time before it is considered stale. It is a companion objective to correctness, ensuring data is not only accurate but also timely.

• Measures Latency: Targets the time between when an event occurs and when it is available for consumers.
• Example Objective: "95% of dashboard data must be less than 5 minutes old."
• Common SLI: The percentile distribution of data latency (p95, p99).

A Data Completeness SLO defines the minimum acceptable percentage of expected data records or fields that must be present and non-null for a dataset to be considered usable. It ensures data pipelines are not silently dropping information.

• Measures Missing Data: Targets gaps in data coverage, such as missing rows from a source or null values in critical columns.
• Example Objective: "99.9% of expected hourly transaction records must be delivered."
• Common SLI: Record arrival rate or null-value rate for key dimensions.

Automated Remediation is the practice of using software systems to automatically detect and resolve common failures or deviations in data pipelines without human intervention. It is a key tactic for preserving error budgets and reducing manual toil.

• Correctness Applications: Automatically quarantining invalid records to a dead-letter queue, triggering a re-run of a failed validation job, or rolling back a flawed schema change.
• Requires Robust Detection: Depends on high-fidelity SLI measurement and clear, safe resolution paths.
• Reduces MTTR: Drastically lowers the Mean Time to Resolution for well-understood failure modes.