GPU Direct Storage

GDS enables a Direct Memory Access (DMA) engine to transfer data straight from NVMe drives or network-attached storage into GPU memory. This completely circumvents the CPU's buffer copy mechanism, eliminating the traditional bounce buffer in system memory. The result is a significant reduction in I/O latency and a dramatic increase in usable bandwidth for large-scale training datasets.

The technology is exposed through the cuFile API, a core component of the CUDA toolkit. This API provides a POSIX-like interface for file operations, allowing developers to read directly from storage into GPU device memory without complex memory management. It supports standard file system operations and is compatible with both local file systems and parallel distributed file systems like Lustre and GPFS.

GDS extends its direct data path over the network by integrating with Remote Direct Memory Access (RDMA) technologies. This allows data to be pulled directly from remote storage servers or NVMe over Fabrics (NVMe-oF) targets into GPU memory without CPU intervention on either the client or server side. It is a critical enabler for disaggregated storage architectures in large-scale AI clusters.

The GDS functionality relies on the nvidia-fs.ko kernel module, which works in tandem with the NVIDIA GPU driver. This module intercepts storage I/O requests and orchestrates the DMA transfer between the storage controller and the GPU. It is designed to handle page-faults and memory pinning automatically, ensuring that GPU memory buffers are properly locked for direct hardware access during the transfer.

To leverage GDS, specific hardware and software alignment is required:

• GPU: NVIDIA Volta architecture or newer (Tesla V100, A100, H100).
• Storage: NVMe SSDs or NVMe-oF targets.
• Network: Mellanox/NVIDIA ConnectX adapters for RDMA.
• File System: A GDS-compatible file system such as ext4, XFS, or a parallel file system with GDS support enabled.
• CUDA: Version 11.4 or later with the cuFile library.

GPU Direct Storage is a foundational component of NVIDIA's Magnum IO suite, which encompasses the entire I/O stack for accelerated computing. It works in concert with GPUDirect RDMA (for network-to-GPU transfers) and GPUDirect P2P (for GPU-to-GPU transfers) to create a fully optimized data movement ecosystem. This integration ensures that data never makes a round-trip through the CPU or system memory at any stage of the pipeline.

A core performance benefit of GPU Direct Storage. In traditional I/O, data is staged in a CPU-attached bounce buffer in system DRAM before being copied to GPU memory. This creates a double-copy penalty and saturates the CPU's memory bandwidth. GPU Direct Storage eliminates this intermediate step, enabling a direct PCIe peer-to-peer transfer from the storage device to the GPU's frame buffer.

The hardware-level mechanism that allows two PCIe endpoints—such as an NVMe drive and a GPU—to transfer data directly without routing through the Root Complex or system memory. This requires BIOS and platform support to enable Access Control Services (ACS) redirection. GPU Direct Storage leverages PCIe P2P to map the storage device's BAR space into the GPU's address domain.