SR-IOV

A Physical Function is a full-featured PCIe function embedded in the SR-IOV-capable NIC. The PF acts as the management and configuration anchor for all associated Virtual Functions. It possesses full configuration capabilities, including the ability to create, provision, and manage VFs. The PF is discovered and managed by the host operating system or hypervisor, and it is the entity that handles link state changes and firmware updates. Only one PF typically exists per physical port, and it is not directly assigned to guest virtual machines or pods.

A Virtual Function is a lightweight, pared-down PCIe function that represents a slice of the physical NIC's resources. Each VF contains only the essential resources for data movement: dedicated transmit and receive queues, DMA channels, and a unique PCI Requester ID. VFs lack full configuration capabilities and cannot be used to modify the underlying physical device. They are designed for direct assignment to a single pod or virtual machine via the VFIO (Virtual Function I/O) kernel framework, providing a dedicated, hardware-isolated data path with minimal CPU overhead.

SR-IOV achieves its performance advantage by completely bypassing the hypervisor's virtual switch and the kernel's networking stack for data plane traffic. A VF is mapped directly into a pod's memory space using the VFIO-PCI driver, granting the workload direct hardware access. This eliminates the context switching and packet copying inherent in software-based bridging. The result is near line-rate throughput, microsecond-level latency, and zero CPU steal for packet processing, making it essential for latency-sensitive AI inference serving and high-frequency trading platforms.

Each VF operates with hardware-enforced isolation from other VFs on the same physical port. The NIC's on-chip switch enforces traffic separation, preventing one noisy neighbor from starving another of bandwidth. Advanced NICs support Quality of Service (QoS) policies configurable via the PF, allowing administrators to set minimum and maximum bandwidth guarantees per VF. This deterministic resource allocation is critical for multi-tenant Kubernetes clusters where AI training jobs must coexist with latency-bound inference serving without mutual interference.

In Kubernetes, the SR-IOV Network Operator automates the lifecycle management of SR-IOV devices. It discovers compatible NICs, configures PFs, and provisions VFs based on declarative SriovNetworkNodePolicy custom resources. The operator integrates with Multus CNI to attach VF-backed interfaces to pods as secondary networks. Key capabilities include:

• Device Plugin: Advertises available VFs to the kubelet as extended resources (e.g., intel.com/sriov_netdevice)
• Configuration Daemon: Applies node-specific VF counts and driver settings
• Policy-Driven: Allows per-node or pool-based VF topology configuration

For optimal performance, SR-IOV VFs must be allocated from a NIC that resides on the same NUMA node as the pod's assigned CPUs and GPU. Cross-NUMA traffic incurs significant latency penalties due to inter-socket communication. The Topology Manager kubelet component, combined with the SR-IOV device plugin, can enforce single-NUMA-node alignment policies. This ensures that a pod requesting a VF, CPU cores, and hugepages receives all resources from the same physical socket, eliminating non-uniform memory access bottlenecks for distributed AI training workloads.