Data Contract

The contract enforces a precise logical structure using typed schemas such as Apache Avro, Protocol Buffers (protobuf) , or JSON Schema. This includes explicit field names, data types (e.g., INT64, TIMESTAMP), and nullability constraints. By defining the schema upfront, the contract prevents training-serving skew and ensures that consumers can deserialize data without ambiguity, eliminating the silent data corruption common in schema-on-read architectures.

Beyond technical types, a data contract binds columns to a shared business vocabulary to resolve semantic ambiguity. A field named status is explicitly defined as 'The lifecycle state of a purchase order as defined by the OMS,' not just a VARCHAR(255). This alignment between data producers and data consumers prevents the misinterpretation of critical metrics and ensures that AI models trained on the data use features with consistent, documented real-world meaning.

The contract codifies measurable quality guarantees that the producer commits to maintaining. These include:

• Freshness: Data will be no older than 15 minutes (max lag).
• Completeness: No more than 0.1% of records will have null values in required fields.
• Accuracy: Revenue figures will reconcile with the general ledger within a 0.01% margin. If these SLOs are breached, the contract triggers automated alerts, preventing data drift from silently degrading downstream model performance.

Data contracts are versioned artifacts (e.g., v1.2.0) that evolve under strict compatibility rules. A producer can add a new optional field without breaking existing consumers (forward compatibility), but removing a required field mandates a major version bump. This discipline allows data lineage tools to track exactly which contract version produced a given dataset, enabling reproducible model training and safe, decoupled evolution of data pipelines.

The contract embeds governance policies directly into the data interface. It specifies which personally identifiable information (PII) columns must be encrypted or tokenized, enforces data residency restrictions (e.g., 'EU user data must remain in Frankfurt'), and defines retention periods. This transforms the contract into a programmable enforcement point for data minimization and purpose limitation, ensuring compliance with GDPR and the EU AI Act before data ever reaches a model.

Every data contract explicitly identifies the data owner (the accountable engineering team) and the communication channel (e.g., a Slack channel or Jira project) for incident response. It defines a Service-Level Agreement (SLA) with escalation paths for when SLOs are violated. This closes the operational loop, ensuring that if a data pipeline breaks and threatens an AI model's accuracy, the consumer knows exactly who to contact and what remediation to expect, moving data reliability from a best-effort to a contractual obligation.

The programmatic validation layer that ensures ingested data conforms to the structure, data types, and constraints defined in a data contract. Schema enforcement acts as the contract's execution arm, rejecting records that violate field-level rules before they enter downstream pipelines.

• Validates column names, data types, nullability, and value ranges
• Prevents corrupt or malformed records from poisoning training datasets
• Often implemented via JSON Schema, Protobuf, or Apache Avro
• Critical for maintaining training-serving consistency in ML systems

A statistical shift in the distribution of input data compared to the training baseline. Data contracts mitigate drift by codifying acceptable statistical bounds—when production data violates these bounds, the contract triggers alerts or blocks ingestion.

• Feature drift: Individual column distributions change
• Schema drift: New fields appear or existing fields change type
• Contracts specify drift thresholds (e.g., PSI < 0.1, KS statistic limits)
• Unchecked drift is a leading cause of silent model degradation

The documented origin and custody chain of a dataset, establishing its authenticity and trustworthiness. Data contracts formalize provenance by requiring producers to declare source systems, collection methodologies, and transformation history.

• Answers: Who created this data? Under what conditions?
• Critical for copyright compliance in AI training datasets
• Supports training data attribution when models exhibit unexpected behavior
• Often represented using W3C PROV standard or custom metadata fields

A structured, human-readable document providing essential context about a dataset—its motivation, composition, collection process, and recommended uses. Data cards complement machine-readable contracts by offering the narrative context that schemas alone cannot convey.

• Originated from Google's Datasheets for Datasets research
• Includes ethical considerations and known biases
• Serves as the human-facing counterpart to a data contract's technical specification
• Increasingly required for AI model transparency documentation

A discrepancy between the data processing logic used during model training and the logic applied during inference. Data contracts prevent skew by enforcing identical transformation rules at both the training data ingestion point and the serving endpoint.

• Common causes: feature engineering divergence, timestamp handling differences
• Contracts codify feature definitions so training and serving pipelines remain synchronized
• A feature store often serves as the enforcement mechanism for contract consistency
• Eliminating skew is a primary motivation for adopting data contracts in MLOps