Kernel Driver

The driver creates a standardized software interface to the unique, vendor-specific NPU hardware. It handles the low-level details of device discovery, initialization, and power state management (e.g., clock gating, DVFS). This abstraction allows higher-level software like the Vendor Runtime to operate without direct knowledge of register maps or chip revision specifics.

• Probes for the device on the system bus (e.g., PCIe, AXI).
• Maps physical device registers and memory into the kernel's address space.
• Exposes a character device (e.g., /dev/npu0) for user-space interaction.

NPU workloads require efficient movement of large tensors and weights. The kernel driver manages the allocation and mapping of memory accessible by both the host CPU and the NPU.

• Allocates contiguous DMA buffers for model weights, inputs, and outputs.
• Manages IOMMU (Input-Output Memory Management Unit) translations to provide the NPU with secure, virtualized addresses.
• Handles cache coherency operations to ensure the CPU and NPU see consistent data.
• Implements memory pinning to prevent the OS from paging out critical workload data.

The driver provides the mechanism for submitting computational graphs or kernels to the NPU for execution. It translates high-level API calls into sequences of hardware commands.

• Maintains command queues (rings) in shared memory.
• Accepts workload descriptors from the user-space Driver API.
• Formats and places commands into the NPU's dispatch unit.
• Manages queue prioritization and preemption for multi-tenant scenarios.

The NPU signals completion of tasks or errors asynchronously via hardware interrupts. The driver's interrupt service routine (ISR) is critical for low-latency response and correct synchronization.

• Registers an ISR for NPU-generated interrupts.
• Services interrupts by reading status registers to determine cause (e.g., job done, error).
• Signals waiting user-space processes or completion fences.
• Handles timeout detection for stalled workloads.

The driver must robustly manage hardware faults, invalid commands, and system errors to prevent kernel panics and ensure system stability.

• Validates all user-submitted commands and pointers for safety.
• Implements watchdog timers to detect hung hardware.
• Performs reset sequences on the NPU core or entire device in case of unrecoverable errors.
• Logs detailed telemetry (via kernel ring buffer) for debugging hardware or driver issues.

To enable optimization and profiling, the driver provides access to the NPU's hardware performance monitoring units (PMUs).

• Exposes ioctl() calls or sysfs nodes to read performance counters (e.g., tensor ops/sec, memory bandwidth, cache hit rates).
• Manages multiplexing of limited physical PMU resources among multiple profiling clients.
• Provides low-overhead timestamping for fine-grained execution tracing.
• Feeds data into system-wide profiling frameworks like perf.

A vendor-specific Software Development Kit provides the foundational libraries, compilers, debuggers, and documentation for programming a hardware accelerator like an NPU. It abstracts the hardware complexity but provides the necessary hooks for low-level control.

• Contains: Compiler toolchains, performance libraries (e.g., cuBLAS, oneDNN), profilers, and emulators.
• Purpose: Enables developers to write portable code that still leverages vendor-specific hardware optimizations.
• Relationship to Driver: The SDK's runtime libraries typically communicate with the device via the kernel driver's Driver API.

The programming interface exposed by the kernel driver to user-space applications. It is the controlled gateway for submitting work, managing memory, and querying device status.

• Functions Include: Allocating device memory (cudaMalloc, clCreateBuffer), submitting command queues or kernels, and synchronizing execution.
• Implementation: Often presented as a userspace library (e.g., CUDA Runtime, Level Zero) that makes system calls to the driver.
• Key Concept: It provides a stable abstraction, allowing the same application code to work across different driver versions and sometimes even hardware generations.

A thin software layer that sits between the kernel driver and the physical hardware registers. It translates generic driver commands into the precise sequence of register writes and MMIO operations required by a specific chip.

• Purpose: Decouples the core driver logic from hardware revision details. A single driver can support multiple chip variants by swapping the HAL.
• Example: In GPU drivers, the HAL manages the specifics of command processor submission, power management sequences, and memory controller initialization.
• Contrast with Driver API: The HAL is internal to the kernel driver, while the Driver API is the external interface.

A vendor-provided userspace library that manages the execution environment and high-level lifecycle of accelerator workloads. It often sits above the Driver API.

• Responsibilities: Device discovery, context management, just-in-time (JIT) compilation of kernels, and managing dependencies between submitted tasks.
• Example: The CUDA Runtime API is a higher-level, more convenient wrapper over the lower-level CUDA Driver API.
• Interaction: The runtime handles complexity like implicit memory transfers and default stream management, ultimately issuing commands via the Driver API to the kernel driver.

A low-level contract that defines how different software components interact at the binary level. For kernel drivers, the ABI is critical for stability.

• Defines: The layout of data structures passed between userspace and kernel space, system call numbers, and calling conventions for driver ioctls.
• Importance: A stable driver ABI allows userspace libraries (like a Vendor SDK) from one software release to work with a slightly different kernel driver version.
• Breaking the ABI requires recompilation of all dependent userspace software, a significant event for system maintainers.

A vendor's non-standard, opaque programming interface and toolchain. This model is common in accelerator ecosystems (e.g., NVIDIA CUDA, Apple Metal) and has direct implications for kernel drivers.

• Kernel Driver Role: The driver is a closed-source binary blob that is the only software component with permission to talk to the hardware. It enforces the vendor's control model.
• Implications: Limits deep performance tuning, complicates open-source OS support, and creates vendor lock-in. The driver's internal mechanisms (scheduling, preemption) are opaque.
• Contrast: An open driver model, as pursued by some communities, exposes driver internals and allows for community-led optimization and debugging.