Model Provenance

The end-to-end documented history of all datasets used to train a model. This includes:

• Origin: Source systems, sensors, or third-party providers
• Transformations: Cleaning, augmentation, and labeling processes applied
• Versioning: Immutable snapshots of the dataset at each pipeline stage

Without data lineage, a model's outputs cannot be traced back to their factual grounding, making regulatory compliance impossible.

The network of data providers, model developers, and tooling vendors that contribute components to a final AI system. Key elements include:

• Base model attribution: Which foundation model was fine-tuned?
• Dependency tracking: All libraries, frameworks, and APIs consumed
• Vendor provenance: The identity and practices of every upstream supplier

This forms the basis for the AI Bill of Materials (AIBOM) , a structured inventory analogous to a software SBOM.

A chronologically ordered, tamper-evident record of every action performed on a model throughout its lifecycle. Critical events logged include:

• Training runs: Hyperparameters, compute resources, and duration
• Evaluation checkpoints: Performance metrics and safety benchmarks at each stage
• Human interventions: Who approved a model for promotion, and when

This trail enables non-repudiation, proving that a specific process produced a specific artifact.

Techniques to embed a persistent, verifiable identifier directly into a model's weights or behavior. This serves two purposes:

• IP Protection: Proving ownership if a model is stolen or leaked
• Provenance Verification: Confirming that a deployed model matches the certified artifact

Watermarks survive fine-tuning and compression, acting as a forensic link back to the original training run and its documented lineage.

The structured body of proof demonstrating that a model meets the essential requirements of a specific regulation, such as the EU AI Act. Provenance records provide:

• Training data copyright compliance: Proof of lawful data acquisition
• Bias and fairness evaluation results: Documented testing against disparate impact metrics
• Risk classification justification: Evidence supporting the model's designated risk tier

Provenance transforms a conformity assessment from a claim into a verifiable fact.

The technical capability to recreate a specific model artifact from its provenance records. This requires:

• Environment pinning: Exact versions of all dependencies and container images
• Data snapshotting: Immutable references to the exact training data used
• Deterministic pipelines: Scripted workflows that produce bitwise-identical outputs

This infrastructure enables forensic debugging of model failures and supports regulatory rollback procedures when a model must be decommissioned.

The documented end-to-end origin, movement, and transformation history of all datasets used to train a model. It tracks how raw data was collected, cleaned, labeled, and versioned before ingestion.

• Provenance Graph: A directed acyclic graph (DAG) mapping data sources to model checkpoints
• Drift Detection: Lineage tools automatically flag when upstream data schemas or distributions change
• Compliance: Essential for demonstrating GDPR data minimization and copyright adherence to auditors

A formal, structured inventory of all software, data, and model components used to construct an AI system. Analogous to a Software Bill of Materials (SBOM) in cybersecurity.

• Component Hashing: Each artifact is cryptographically hashed to ensure integrity
• Dependency Tree: Maps transitive dependencies, including pre-trained weights and fine-tuning scripts
• Vulnerability Scanning: Enables automated checks against known CVEs in underlying frameworks like PyTorch or TensorFlow

A structured transparency document detailing a machine learning model's intended use, performance benchmarks, and limitations. Proposed by Google Research, it is now a de facto standard for model provenance.

• Intended Use: Explicitly defines out-of-scope applications to limit liability
• Evaluation Metrics: Reports disaggregated performance across demographic subgroups
• Data Provenance: Summarizes training data composition and known biases

The technique of embedding a hidden, persistent identifier into a model's weights to prove intellectual property ownership. This is a critical anti-theft mechanism in the model supply chain.

• White-Box Watermarking: Embeds a secret pattern directly into the weight matrices during training
• Black-Box Watermarking: Triggers predefined outputs via a backdoor key in the input
• Robustness: Must survive fine-tuning, pruning, and model compression to be forensically valid

An immutable, chronological record of all assessments and validations performed by an external auditor on a vendor's AI system. It provides non-repudiation for regulatory filings.

• Cryptographic Chaining: Uses Merkle trees to prevent log tampering
• Conformity Evidence: Stores proofs of EU AI Act compliance checks
• Continuous Monitoring: Integrates with CI/CD pipelines to log every model promotion event

A disclosure detailing the training data, compute resources, and capabilities of a general-purpose AI model to meet regulatory obligations under the EU AI Act.

• Compute Threshold: Reports total FLOPs to determine if systemic risk thresholds are triggered
• Data Sources: Enumerates all web crawls, licensed corpora, and synthetic data used
• Dangerous Capability Benchmarks: Discloses results on CBRN and cyber-offense evaluations