DPU

The DPU integrates a high-speed NIC with a programmable data path to offload the entire networking data plane. This includes OVS (Open vSwitch) acceleration, VXLAN encapsulation/decapsulation, and TCP/IP segmentation offload. By processing packets directly on the DPU's dedicated processors, it eliminates the CPU overhead of virtual switching, delivering line-rate throughput (up to 200Gb/s or 400Gb/s) with deterministic low latency. This is the foundation of software-defined networking in modern cloud and AI clusters.

DPUs present emulated VirtIO block and network devices to virtual machines, completely bypassing the hypervisor's software emulation layer. For storage, VirtIO-NVMe emulation exposes a standards-compliant NVMe controller to the guest VM. The DPU translates these I/O requests into physical NVMe commands over NVMe-oF (NVMe over Fabrics) or local PCIe. This provides bare-metal-like storage performance to VMs while the host CPU remains entirely isolated from the storage I/O path.

A DPU creates a hard security boundary between the infrastructure domain and the tenant domain. The DPU's own OS and control plane run on isolated ARM cores, invisible to the host. This enables zero-trust architectures where:

• The host cannot inspect or tamper with network encryption
• IPsec and TLS termination happen on the DPU
• Hardware root of trust verifies firmware integrity on boot
• Stateful firewall and micro-segmentation policies are enforced in hardware, not in a vulnerable hypervisor

DPUs are critical enablers for GPU clusters by offloading RoCE (RDMA over Converged Ethernet) and InfiniBand transport processing. The DPU handles the complex RDMA connection state, memory registration, and transport retry logic. With GPUDirect RDMA, the DPU facilitates direct data transfers between a remote GPU's memory and a local GPU's memory across the network, completely bypassing the CPU and system memory on both nodes. This is essential for scaling distributed AI training with NCCL.

Unlike fixed-function NICs, a DPU's packet processing pipeline is software-defined. Using the P4 language or NVIDIA's DOCA (Data Center Infrastructure-on-a-Chip Architecture) SDK, engineers can program custom match-action tables in the DPU's switching ASIC. This allows for in-line, line-rate telemetry generation, custom load balancing, intrusion detection pattern matching, and novel congestion control algorithms without consuming a single host CPU cycle.

The DPU functions as a full NVMe-oF initiator and target, disaggregating storage from compute. As an initiator, it connects to remote NVMe drives over RoCE or TCP fabrics, presenting them as local block devices to the host. As a target, it exposes local NVMe SSDs to the network. The entire NVMe command set, including queue management and data transfer, is processed on the DPU. This enables composable disaggregated storage architectures with sub-10-microsecond access latency.

GPUDirect RDMA is the critical protocol that enables a DPU to transfer data directly to and from GPU memory over the network. Without this, a data transfer would copy from GPU to pinned system memory, involve the CPU for protocol processing, and then move to the NIC. The DPU, acting as a peer PCIe endpoint, uses GPUDirect to read GPU memory directly, apply encryption or compression via its onboard accelerators, and stream the result onto the wire. This eliminates the CPU from the critical path, freeing it entirely for application logic.

The NVIDIA BlueField-3 DPU is the current industry benchmark. It integrates 16 ARM Cortex-A78 cores, a 400 Gbps ConnectX-7 network interface, and dedicated accelerators for IPsec, TLS, NVMe-oF, and regular expression processing. It runs NVIDIA DOCA, a software framework that abstracts the hardware accelerators. In a sovereign AI stack, BlueField-3 enables bare-metal provisioning via the Redfish API, secure zero-touch provisioning, and real-time telemetry streaming via DCGM, all without consuming host CPU cycles.