Dynamic Library

Unlike static libraries, which are embedded into an executable at compile time, dynamic libraries are linked by the operating system's loader when the program is launched or during execution via functions like dlopen(). This defers symbol resolution, allowing:

• Multiple applications to share a single copy of the library code in memory.
• The library to be updated or patched without recompiling the main executable.
• Conditional loading of functionality based on runtime environment or hardware detection (e.g., loading NPU-specific kernels only if the accelerator is present).

A primary advantage is the reduction of physical memory footprint. The text segment (executable code) of a dynamic library is loaded into RAM once and mapped into the virtual address space of every process that uses it. This is critical in embedded and accelerator contexts where:

• Multiple processes use the same vendor runtime (e.g., CUDA, ROCm, or a proprietary NPU runtime).
• System memory is constrained, as on edge devices or within an NPU's host-side memory hierarchy.
• Read-only code segments can be safely shared across processes, while each process gets its own private copy of writable data segments.

Dynamic libraries employ naming and symbol versioning to manage compatibility. The Application Binary Interface (ABI) defines the low-level contract between the library and the executable.

• Soname: A versioned library name (e.g., libvendor.so.1) embedded in executables to ensure they load a compatible version.
• Symbol Versioning: Allows multiple versions of the same function to coexist within a single library, letting older programs link to the legacy version while new programs use the updated one.
• This is essential for vendor SDKs, where the underlying hardware driver or NPU firmware may change, but existing compiled applications must continue to run without modification.

Beyond automatic loading at program start, programs can explicitly load libraries at runtime using the Dynamic Linking API (dlopen, dlsym, dlclose). This enables plugin architectures and hardware abstraction:

• A program can probe for an NPU's presence and load the appropriate vendor-specific optimization library (libnpuvendor.so).
• Function pointers for hardware intrinsics or proprietary APIs can be retrieved via dlsym().
• This allows a single binary (a fat binary in spirit) to support multiple accelerator backends, falling back to a CPU implementation if no compatible dynamic library is found.

Because a dynamic library cannot know its final load address in advance, it must be compiled as Position-Independent Code (PIC). PIC uses relative addressing (e.g., PC-relative jumps) and a Global Offset Table (GOT) to reference global data.

• The linker creates relocation entries in the library. The dynamic loader performs these relocations, patching addresses when the library is loaded into memory.
• This process adds a small overhead at load time but is essential for security (Address Space Layout Randomization) and for allowing the same library code to be loaded at different virtual addresses in different processes.

Dynamic libraries can have their own dependencies, forming a graph. The dynamic linker (e.g., ld.so on Linux) resolves this graph, loads all required libraries, and performs symbol resolution and relocations.

• Tools like ldd or objdump -p list a library's DT_NEEDED entries.
• Dependency hell can occur with conflicting or missing versions. Vendor toolchains often bundle specific versions of their runtime libraries to avoid this.
• In NPU programming, the main application may depend on a high-level Vendor SDK dynamic library, which in turn depends on a lower-level driver API library and ultimately the kernel driver.

A static library (often with a .a or .lib extension) is an archive of pre-compiled object code that is linked into an executable at compile time. Unlike a dynamic library, the code becomes a permanent, inseparable part of the final binary.

• Key Difference: Eliminates runtime dependency but increases final binary size.
• Use Case: Deploying a self-contained application where versioning and shared updates are not required.
• NPU Context: Vendor SDKs often provide static libraries for critical, performance-sensitive kernels to be embedded directly into the accelerator's firmware.

The Application Binary Interface (ABI) is a low-level contract that defines how binary code interacts with an operating system or other binaries. It specifies calling conventions, system call numbers, register usage, and data structure layout.

• Critical for Dynamic Libraries: Ensures binary compatibility between a dynamically linked library and the executable that loads it.
• Vendor-Specific: NPU runtime libraries and drivers must adhere to a specific ABI for the host CPU architecture (e.g., x86-64, ARM64).

A vendor runtime is a proprietary software library, often distributed as a dynamic library (*.so, *.dll), that manages the execution environment for a hardware accelerator. It handles:

• Device initialization and context management.
• Memory allocation and data transfer between host and NPU.
• Kernel scheduling and execution on the accelerator cores.
• Synchronization and error handling. The application links to this runtime, which in turn communicates with the lower-level kernel driver.

A Hardware Abstraction Layer (HAL) is a software layer that provides a uniform, standardized interface to hardware-specific functionalities. It sits between the high-level application or framework (e.g., TensorFlow, PyTorch) and the vendor runtime or driver.

• Purpose: Insulates framework code from the intricacies of different NPU vendors' SDKs.
• Implementation: Often compiled into a dynamic library that can be swapped based on the detected hardware.
• Benefit: Enables a single codebase to target multiple accelerator types (e.g., NVIDIA, AMD, Intel NPUs).

ELF Sections are contiguous segments within an Executable and Linkable Format (ELF) file, which is the standard binary format on Unix-like systems (and often used for NPU binaries). Key sections relevant to dynamic libraries include:

• .text: Contains the executable machine code.
• .data & .bss: Hold initialized and uninitialized static data.
• .dynamic: Contains metadata essential for dynamic linking (list of required libraries, symbol tables).
• .plt & .got: Facilitate position-independent code and global offset tables for external function calls. The dynamic linker uses these sections to load and relocate the library in memory.

A fat binary (or multi-architecture binary) is a single executable file that contains machine code and data for multiple hardware architectures. This is distinct from a dynamic library's single-architecture, loadable code.

• Mechanism: Bundles separate code images (e.g., for x86 CPU and a vendor NPU) with a header describing each.
• Runtime Selection: The OS or loader examines the binary and executes the native code for the detected hardware.
• NPU Use: Allows a single application binary to contain optimized kernels for different NPU generations or fallback CPU code, simplifying deployment.