Data Provenance

Provenance captures a tamper-proof, chronological record of every action performed on a clinical data element. This includes creation, access, modification, and deletion events.

• Uses W3C PROV standard to represent entities, activities, and agents
• Enables forensic reconstruction of a data point's entire lifecycle
• Critical for demonstrating HIPAA audit control compliance to regulators

Two distinct capture methodologies serve different operational needs.

• Retrospective provenance answers "What happened?" by reconstructing lineage from logs and metadata after the fact
• Prospective provenance answers "What should happen?" by capturing a planned workflow or computational recipe before execution
• Clinical research often requires both: the intended protocol and the actual execution record

Provenance records must capture responsibility at multiple levels of abstraction to be meaningful in a healthcare context.

• System-level: Which interface engine or FHIR server performed the transformation
• User-level: Which specific clinician or data steward approved a change
• Algorithm-level: Which version of an NLP model extracted a diagnosis code from a radiology report
• This layered attribution is essential for resolving disputes in prior authorization denials

When an HL7 v2 message is mapped to FHIR, provenance tracks the exact logic applied.

• Records the source field (e.g., PID-3) and the target element (e.g., Patient.identifier)
• Captures any data cleansing rules applied, such as date normalization or code system translation from ICD-9 to ICD-10-CM
• Preserves the original raw value alongside the transformed value to prevent information loss and enable rollback

Provenance chains are secured using cryptographic hashing to detect unauthorized tampering.

• Each provenance entry includes a SHA-256 hash of the data payload and a pointer to the previous entry's hash
• Forms a verifiable chain of custody analogous to blockchain structures
• Allows any downstream consumer to independently validate that a clinical document has not been altered since its attested creation by the author

Provenance metadata informs the trustworthiness score assigned to ingested data.

• TOFU automatically trusts data from a new source until a conflict or anomaly is detected
• Explicit trust requires a governance board to pre-authorize specific sending systems and their data domains
• A robust provenance system enables dynamic trust calculation based on the completeness of the lineage chain and the reputation of the originating authoring system

While often used interchangeably, these terms represent distinct scopes of metadata tracking:

• Data Provenance: A broad, contextual record of a data asset's entire lifecycle, including its origins, ownership, custody, and the processes that acted upon it. Provenance answers 'who, what, when, where, and why'—providing the evidentiary chain for regulatory compliance and trust.
• Data Lineage: A narrower, technical subset of provenance focused specifically on the forward and backward flow of data through transformation pipelines. Lineage maps how a specific output field was derived from input fields across ETL jobs, API calls, and model inferences. In clinical interoperability, provenance provides the legal and clinical context, while lineage provides the technical debugging map for data engineers.

A cryptographic technique used to validate the authenticity and integrity of provenance assertions. When a system records a provenance event, it generates a hash of the event data and encrypts it with the system's private key, creating a digital signature. Any downstream consumer can use the corresponding public key to verify that the provenance record has not been altered since it was signed and that it originated from the claimed source. In healthcare, this is critical for C-CDA document provenance, where a signed document must provably link back to the authoring clinician and organization without any post-hoc tampering.