Inter-Process Communication (IPC)

A pipe is a unidirectional, byte-oriented communication channel, typically used for sequential data flow between related processes (e.g., a parent and child). Data written to the write end is buffered by the kernel and read from the read end.

• Anonymous Pipes: Created with the pipe() system call, exist only for the lifetime of the processes and are unnamed. Commonly used for shell command chaining (e.g., ls | grep .txt).
• Named Pipes (FIFOs): Have a filesystem entry, allowing unrelated processes to communicate. Created with mkfifo().
• Characteristics: Simple, kernel-buffered, but limited to one-way communication and typically used with related processes.

A message queue is a kernel-managed, structured messaging system where processes can send and receive discrete messages (packets of data) asynchronously. Messages are stored in a queue until read.

• Key Features: Messages have a defined type and priority. The queue persists until explicitly deleted, allowing communication between unrelated processes.
• Synchronization: Reading is typically blocking by default if the queue is empty, providing built-in flow control.
• Use Case: Suitable for structured, asynchronous communication where message ordering and persistence are important, often used in System V IPC and POSIX message queues.

Shared memory is the fastest IPC mechanism, allowing multiple processes to map the same region of physical memory into their address spaces. This enables direct data access without kernel intervention for each read/write.

• Performance: Eliminates data copying between kernel and user space, offering near-native memory access speeds.
• Synchronization Critical: Because processes access memory directly, explicit synchronization primitives like semaphores or mutexes are required to prevent race conditions and ensure data consistency.
• Setup: Involves creating a shared memory segment (shmget() or shm_open()) and attaching it to the process's address space (shmat() or mmap()).

A semaphore is a synchronization primitive, not a data transfer mechanism. It is used to control access to a shared resource (like a shared memory region) by multiple processes, preventing race conditions.

• Operation: A semaphore is a counter that supports two atomic operations: wait (decrement) and signal (increment). If the counter is zero, a wait operation will block the process.
• Types: Binary semaphores (mutex-like, value 0 or 1) and counting semaphores (allow a predefined number of concurrent accesses).
• IPC Role: Often used in conjunction with shared memory to guard critical sections, ensuring that only one process modifies the shared data at a time.

Sockets are a generalized, bidirectional communication endpoint that can be used for IPC between processes on the same machine (Unix domain sockets) or across a network (Internet sockets).

• Unix Domain Sockets: Use a filesystem path as an address. Extremely efficient for local IPC, supporting both stream (TCP-like, reliable, connection-oriented) and datagram (UDP-like, connectionless) modes.
• Characteristics: Provide full-duplex communication, support for multiple connection protocols, and can pass file descriptors between processes (a unique capability).
• Use Case: The most flexible IPC mechanism, ideal for client-server architectures, even on a single host, due to their standardized API and network transparency.

Memory-mapped files allow a process to treat file contents as a byte array in memory. For IPC, multiple processes can map the same file, creating a shared memory region backed by the filesystem.

• Mechanism: The mmap() system call maps a file (or anonymous memory) into the process's virtual address space. Updates to the memory are eventually flushed to the file.
• Persistence: Provides a natural persistence model, as the shared state is stored on disk.
• Synchronization: Like shared memory, requires explicit synchronization. The msync() call can be used to control when changes are written to disk. Useful for large, persistent data structures shared between processes.

A hardware feature that allows peripherals (like NPUs, network cards, or storage controllers) to read from and write to system memory without continuous CPU intervention. This is critical for high-throughput IPC and data movement in accelerated computing.

• Purpose: Offloads data transfer tasks from the CPU, freeing it for computation.
• IPC Role: Enables efficient bulk data sharing (e.g., moving model weights or inference results) between a host process and an accelerator, forming the hardware backbone for shared memory IPC.

An OS-level virtualization method that packages an application and its dependencies into an isolated, lightweight container. Containers share the host OS kernel but run in separate user-space instances.

• IPC Context: Containers provide process isolation, but IPC mechanisms (like Unix domain sockets or shared volumes) are used for communication between containers or between a container and the host.
• Deployment Impact: Essential for packaging and deploying inference servers, model microservices, and other components that need to communicate via IPC in a reproducible environment.

A low-level contract between software components that defines how they interact at the machine code level. It specifies calling conventions, data structure layout, register usage, and system call numbers.

• IPC Relevance: For IPC using shared libraries or system calls, a stable ABI ensures that binaries compiled at different times can interoperate correctly.
• System Integration: When deploying optimized NPU kernels or runtime libraries, the ABI must be compatible between the compiler, the OS, and the hardware SDK.

An instruction or sequence guaranteed to execute indivisibly relative to other threads or processes. No intermediate state is observable, ensuring data consistency in concurrent environments without locks.

• IPC Use Case: Fundamental for implementing synchronization primitives (like semaphores, mutexes) used in shared memory IPC to prevent race conditions.
• Hardware Support: Modern CPUs and NPUs provide atomic instructions (e.g., compare-and-swap) for coordinating access to shared data structures in memory.

The set of techniques and policies used to manage system resources to meet specific performance targets for latency, throughput, and reliability.

• IPC Connection: IPC mechanisms are often a bottleneck. QoS policies can prioritize IPC traffic for critical services (e.g., real-time inference requests) over background tasks.
• Deployment Consideration: In NPU-accelerated systems, QoS ensures that data movement IPC between CPU and NPU does not starve other system processes, maintaining overall system responsiveness.

A secure, isolated area within a main processor that provides confidentiality and integrity for code execution and data, even if the host OS is compromised.

• Secure IPC: Enables confidential IPC where sensitive data (e.g., model parameters, private inputs) can be passed between processes with hardware-enforced encryption and access control.
• Use Case: Critical for multi-tenant AI inference deployments or federated learning edge nodes where models and data must be protected from other processes on the same hardware.