Fat Binary

A fat binary (also known as a universal binary or multi-architecture binary) packages compiled code for distinct Instruction Set Architectures (ISAs) into a single file. This is achieved by concatenating multiple ELF sections or Mach-O segments, each tagged for a specific target (e.g., arm64, x86_64, NPU_Vendor_A). The operating system's loader reads the file header, identifies the compatible architecture slice, and executes only the relevant machine code. This eliminates the need for separate distribution channels for different hardware.

Each architecture slice within the binary is compiled and optimized for its specific target. This allows developers to leverage vendor-specific intrinsics, SIMD instructions, and hardware accelerators like NPUs for each platform. For example:

• An arm64 slice can use ARM's NEON instructions.
• An x86_64 slice can use AVX-512.
• An NPU-specific slice can contain kernels compiled via a vendor SDK to use tensor cores. The fat binary format preserves these hardware-aware optimizations within a single deployable artifact.

Execution is managed by the system's dynamic linker/loader (e.g., ld.so on Linux, dyld on macOS). The loader:

1. Parses the fat binary's header to discover available architecture slices.
2. Matches a slice to the host machine's CPU/accelerator architecture.
3. Loads only the matched slice's .text (code) and .data sections into memory.
4. Performs any necessary relocations for that specific slice. Unused slices are ignored, incurring no runtime overhead. The Application Binary Interface (ABI) for the selected slice governs calling conventions and system interactions.

Creating fat binaries requires a toolchain capable of cross-compilation and multi-architecture linking. Key components include:

• Cross-compilers that generate object files for different target ISAs from a single host machine.
• A linker (e.g., ld) that accepts multiple -arch flags or uses a linker script to combine object files into a single multi-architecture binary.
• Vendor toolchains (like NVIDIA's nvcc with fatbinary support) that can embed both GPU and CPU code. Build systems like CMake or Meson are configured to compile the same source code multiple times with different architecture flags before the final linking stage.

The primary trade-off is increased file size, as the binary contains redundant code for each architecture. This is often acceptable for several reasons:

• Simplifies deployment and distribution: one file works on all supported platforms.
• Reduces complexity in app stores and update systems.
• The size penalty is often marginal compared to total application assets (images, videos). For resource-constrained environments (e.g., embedded systems), thin binaries (single-architecture) may be preferred. Fat binaries are common in desktop/mobile apps (macOS Universal 2) and accelerator programming (CUDA fatbin).

Fat binaries are critical in heterogeneous computing, especially with NPUs and GPUs:

• CUDA Fatbin: Contains PTX (portable intermediate code) and multiple cubin (binary) versions for different GPU compute capabilities.
• OpenCL: Programs can be distributed as source or intermediate IR, with JIT compilation to the target device; fat binary concepts apply to caching these compiled kernels.
• NPU SDKs: Vendor SDKs may output a fat binary containing CPU host code and one or more optimized NPU kernels targeting different generations or configurations of the accelerator, managed by a vendor runtime.

Game engines and publishers use fat binaries to simplify distribution for multi-platform titles, especially on consoles with iterative hardware.

• A game disc or download for a platform like PlayStation 5 may contain executable code optimized for both standard and Pro hardware modes (if available).
• The engine's runtime or system SDK detects the specific hardware capabilities (CPU cores, GPU TFLOPS) and selects the appropriate performance profile from the binary.
• This avoids needing separate SKUs for different console variants and future-proofs the title for mid-generation hardware refreshes.

Fat binaries are created by toolchain utilities that merge single-architecture object files.

• The lipo tool on macOS is the canonical example: lipo -create -output universal_binary x86_binary arm64_binary.
• On Linux, the GNU Binutils objcopy tool can be used to combine ELF sections from multiple binaries into a single multi-arch ELF file.
• The linker (ld) is often invoked with architecture-specific flags and object files to generate the final fat binary in one step.
• Inspection is done with tools like lipo -info, file, or objdump to list the contained architectures.

A low-level interface specification that defines how binary code interacts with an operating system or other programs. For a fat binary to function, the contained architectures must adhere to compatible ABIs for the target system.

• Defines: Calling conventions, system call numbers, data structure layout, and object file formats.
• Ensures binary compatibility between modules compiled by different toolchains for the same OS/architecture.
• Critical for fat binaries, as each embedded architecture-specific slice must conform to the host OS's ABI to execute correctly.

A compiler that runs on one computer architecture (the host) but generates executable code for a different architecture (the target). This tool is essential for creating the individual architecture-specific binaries that are later combined into a fat binary.

• Example: Using an x86_64 Linux machine to compile code for an arm64 NPU or an aarch64 mobile SoC.
• Outputs object files and libraries in the target's instruction set and file format.
• Enables developers to build for heterogeneous hardware (e.g., CPU + NPU) from a single development environment.

Contiguous segments within an Executable and Linkable Format (ELF) binary file that contain specific types of data. A fat binary for Unix-like systems often packages multiple complete ELF files or distinct ELF sections for each architecture.

• Common Sections: .text (executable code), .data (initialized variables), .rodata (read-only data), .bss (uninitialized data).
• Fat Binary Mechanics: The system loader reads the fat binary header, identifies the appropriate architecture slice (a full ELF or set of sections), and loads it.
• Linker scripts are used to control the placement and organization of these sections during the creation of each architecture's binary.

Two methods of combining libraries with an executable, with direct implications for fat binary size and portability.

• Static Linking: Library code is copied directly into the final executable at compile time.
  • Pros: Creates a self-contained binary; no external dependencies.
  • Cons: Increases binary size—a significant concern for fat binaries containing multiple architectures.
• Dynamic Linking: The executable contains references to shared libraries (.so, .dylib) loaded at runtime.
  • Pros: Dramatically reduces fat binary size; allows library updates without recompiling the main app.
  • Cons: Requires compatible shared libraries to be present on the target system.

A vendor-provided software library that manages the execution environment for hardware accelerators like NPUs. A fat binary designed for a system with an NPU must correctly interface with this runtime for the accelerator slice.

• Responsibilities: Device initialization, memory allocation (host/device), kernel submission, synchronization, and power management.
• Fat Binary Integration: The NPU-specific code within the fat binary contains calls to the vendor runtime's API (e.g., for kernel launches).
• Deployment Note: The correct vendor runtime must be installed on the target system for the NPU code path to execute successfully.

Apple's specific implementation of the fat binary concept, historically used for PowerPC/Intel transitions and currently for x86_64/arm64 (Apple Silicon) support. It uses the Mach-O file format instead of ELF.

• Mechanism: A Mach-O file with the FAT_MAGIC header containing multiple architecture slices.
• Tooling: Created using the -arch flag with Apple's clang/llvm toolchain (e.g., -arch x86_64 -arch arm64).
• System Integration: macOS and iOS's kernel (dyld loader) natively recognizes and selects the correct slice. This is a prime, real-world example of fat binaries enabling seamless architectural transitions.