W3C PROV

Defines the fundamental building blocks for representing provenance as a graph of Entities, Activities, and Agents.

• Entity: A physical, digital, or conceptual thing (e.g., a dataset, a file, a row in a database).
• Activity: An action that generates or modifies entities (e.g., a data cleaning script, a model training run).
• Agent: An entity that bears responsibility for an activity (e.g., a user, a software service, an organization). These three types are linked by relationships like wasGeneratedBy, wasAttributedTo, and wasDerivedFrom to form a complete provenance chain.

A formal OWL2 Web Ontology Language encoding of the PROV Data Model, enabling semantic reasoning and linked data integration.

• Maps PROV concepts to RDF classes and properties for direct use in knowledge graphs.
• Allows provenance assertions to be queried using SPARQL alongside domain-specific data.
• Facilitates logical inference; for example, if an entity was generated by an activity that used another entity, a reasoner can infer a derivation relationship.
• Provides a bridge between raw provenance logs and enterprise knowledge graph grounding.

A compact, functional-style syntax designed for writing provenance assertions in a way that is both human-readable and machine-parsable.

• Uses a simple relation(subject, object) format, e.g., wasGeneratedBy(ex:report1, ex:compile_activity, 2024-05-01T10:00:00Z).
• Ideal for illustrating examples in documentation, creating test fixtures, and debugging provenance graphs without the verbosity of RDF/XML or JSON-LD.
• Serves as the canonical representation in the W3C specification itself, making it the clearest way to learn the model.

A formal set of validity rules that define a consistent and well-formed provenance graph, going beyond the basic data model.

• Specifies structural constraints (e.g., an entity cannot be generated by more than one activity).
• Defines typing rules and event ordering constraints to prevent logical contradictions in provenance chains.
• Enables automated validation of provenance documents to ensure they are not just syntactically correct but also semantically sound.
• Critical for building reliable, auditable systems where provenance integrity is non-negotiable.

PROV data can be serialized into multiple web-native formats for interoperability across different systems.

• PROV-JSON (JSON-LD): A JSON-based representation that is lightweight, easy to parse in web applications, and compatible with modern APIs and NoSQL document stores.
• PROV-XML: An XML schema for environments that rely on XML toolchains, such as enterprise service buses and SOAP-based services.
• Both serializations represent the same underlying provenance graph, allowing seamless exchange between a Python data pipeline and a Java enterprise archive system.

A mechanism for packaging multiple provenance descriptions together, each with its own named scope, to handle complex, distributed systems.

• A Bundle is a named set of provenance statements, allowing you to assert provenance about provenance.
• Enables a data aggregator to include both its own transformation activities and the original provenance records from upstream data providers in a single package.
• Solves the problem of provenance collision when integrating data from multiple sources, each with their own agents and activities.
• Essential for building a federated audit trail across organizational boundaries.

A directed acyclic graph (DAG) that visually represents the historical dependencies and transformations of a data artifact. In W3C PROV, this graph is constructed from three core node types:

• Entities: The physical or digital artifacts being described
• Activities: The processes that generate or modify entities
• Agents: The actors or systems responsible for activities

The graph enables root-cause analysis by tracing backward from a suspicious output to its originating inputs, and impact assessment by propagating forward from a corrupted source to all affected downstream assets.

A hardware-rooted mechanism where a Trusted Execution Environment (TEE) digitally signs a statement verifying that specific code executed on specific data without tampering. When combined with W3C PROV, attestation provides cryptographic proof that a recorded provenance chain is authentic.

• The TEE generates a hardware signature binding the computation to the provenance record
• This transforms PROV from a self-asserted log into a verifiable claim
• Critical for regulated industries requiring non-repudiation of data transformations

A W3C standard for tamper-evident, cryptographically verifiable digital credentials that use Decentralized Identifiers (DIDs) to represent claims. In a provenance context:

• An organization can issue a verifiable credential attesting to the authenticity of a dataset
• The credential references the PROV-O ontology to describe who generated the data and under what conditions
• Verifiers can cryptographically validate the credential without contacting the issuer
• Creates a trust framework where provenance claims are independently verifiable rather than self-asserted

The operational practice of tracking and visualizing the complete lifecycle of data as it flows from origin through transformations to final destination. W3C PROV provides the standardized vocabulary for expressing lineage:

• Column-level lineage: Tracking which specific fields contributed to a derived value
• Pipeline lineage: Capturing the sequence of ETL jobs and model inferences
• Business lineage: Mapping technical flows to business glossary terms for regulatory compliance

Tools like OpenLineage implement PROV-compatible metadata collection across Spark, Airflow, and dbt pipelines.

An extension of the Software Bill of Materials concept that inventories the datasets, models, and training pipelines used to create an AI system. W3C PROV serves as the underlying data model for expressing:

• Which training datasets were used and their provenance
• The sequence of fine-tuning operations applied to a base model
• Human feedback incorporated through RLHF processes
• All third-party components in the AI supply chain

This enables organizations to respond to regulatory inquiries about model origins with a complete, standards-based provenance trail.