Inferensys

Glossary

FHIR Bulk Data Access

An HL7 FHIR specification defining an asynchronous, RESTful API for exporting large, flat datasets of patient-level data from a server, designed for population-level analytics and machine learning.
Large-scale analytics wall displaying performance trends and system relationships.
POPULATION-LEVEL DATA EXPORT

What is FHIR Bulk Data Access?

A FHIR specification defining an asynchronous API for exporting large, flat datasets of patient-level data from a server, designed for population-level analytics and machine learning.

FHIR Bulk Data Access, formally the FHIR Asynchronous Request Patterns specification, defines a standardized API for exporting large, flat datasets of patient-level data from a FHIR server. Unlike transactional FHIR APIs that retrieve individual resources, this mechanism is purpose-built for population-level analytics, machine learning, and quality reporting by generating compressed NDJSON files of an entire patient population in a single asynchronous operation.

The workflow operates via a kick-off request that creates a long-running job, returning a Content-Location header for polling. Once complete, the server provides a manifest of downloadable files organized by resource type (e.g., Patient, Observation, Condition). Authentication leverages SMART Backend Services for server-to-server OAuth 2.0 authorization, ensuring secure, unattended data extraction for approved analytical pipelines.

FHIR BULK DATA ACCESS

Core Characteristics of Bulk Data Access

The FHIR Bulk Data Access specification defines an asynchronous, population-level export paradigm distinct from individual patient queries. It is engineered for analytics and machine learning, not real-time transactional workflows.

02

Flat File Output (NDJSON)

The specification mandates output as Newline Delimited JSON (NDJSON) , where each line is a valid JSON object representing a single FHIR resource. This format is critical for population-level analytics because it is:

  • Streamable: Parsers can process files line-by-line without loading the entire dataset into memory.
  • Splittable: Large files can be trivially sharded across distributed compute nodes (e.g., Apache Spark).
  • Appendable: New data can be added without rewriting the entire file structure. This contrasts sharply with the standard FHIR Bundle resource, which requires in-memory parsing of a single, deeply nested JSON object.
03

System-Level vs. Patient-Level Export

Bulk Data Access defines two primary scoping mechanisms:

  • System-Level Export ([base]/$export): Exports all data from all patients across the entire server. This is the most common mode for building a data warehouse or training a population health model.
  • Patient-Level Export ([base]/Patient/$export): Exports all data for a specific, pre-defined cohort of patients. The patient list is typically provided via a Group resource, allowing an institution to export only the data relevant to a specific clinical trial or research query. Both operations filter data based on the authenticated client's scopes, enforcing privacy at the API layer.
05

Resource Type Filtering

Clients can scope exports to specific clinical domains by specifying a _type parameter. For example, _type=Observation,Condition,MedicationRequest exports only lab results, diagnoses, and prescriptions. This is vital for machine learning pipelines that require targeted feature engineering:

  • A radiology AI model might export only ImagingStudy and Observation resources.
  • A pharmacovigilance algorithm might export only MedicationRequest and AdverseEvent. This granular filtering drastically reduces storage costs and processing time compared to exporting the entire patient record.
06

De-Identification via Export

The specification includes a patient parameter that, when set to Patient/$export, allows a server to export a de-identified data set. The server is responsible for applying de-identification rules (e.g., stripping direct identifiers, date shifting) before generating the NDJSON files. This creates a privacy-preserving pipeline where a trusted intermediary can export a limited data set for a research organization without exposing Protected Health Information (PHI), directly supporting federated learning architectures where raw data cannot leave the clinical boundary.

FHIR BULK DATA ACCESS

Frequently Asked Questions

Clear answers to common questions about the FHIR Bulk Data Access specification, designed for health IT architects and interoperability engineers building population-level analytics and machine learning pipelines.

FHIR Bulk Data Access is an HL7 specification defining an asynchronous, flat-file-based API for exporting large, patient-level datasets from a FHIR server, designed specifically for population-level analytics and machine learning rather than individual patient care. Unlike the standard FHIR RESTful API, which is optimized for point-of-care transactional queries, Bulk Data Access uses the FHIR Asynchronous Request Pattern to initiate long-running export jobs. The client requests a kick-off, the server returns a polling URL, and once the job completes, the server provides a manifest of NDJSON (Newline Delimited JSON) files for download. This flat-file format is critical for loading data into analytical platforms like Spark or TensorFlow. The specification defines three core export operations: System-level Export (all patients), Patient-level Export (a specific cohort), and Group-level Export (a pre-defined cohort managed via the Group resource). Security is enforced through SMART Backend Services Authorization, where a pre-authorized client presents a signed JWT to obtain an access token without direct user interaction.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.