Inter-Process Communication (IPC)

Shared Memory is an IPC mechanism where multiple processes access the same region of physical memory, allowing for extremely high-speed data exchange. It is the fastest IPC method because it avoids kernel involvement and data copying after the initial setup.

• Key Concept: Processes map a common memory segment into their own address space.
• Synchronization Required: Since memory is shared, processes must use semaphores, mutexes, or other synchronization primitives to prevent race conditions and ensure data consistency.
• Use Case: Ideal for high-performance computing, real-time systems, and when large volumes of data need to be passed frequently between co-located processes.

Message Queues provide asynchronous, decoupled communication where messages are stored in a queue by a sender and later retrieved by a receiver. Messages are typically handled in a First-In-First-Out (FIFO) order, though some systems support prioritization.

• Core Features: Messages are discrete packets with a defined structure. The queue acts as a buffer, decoupling the timing of production and consumption.
• Reliability: Many message queue implementations offer persistence, ensuring messages survive process or system restarts.
• Use Case: Foundational for Message-Oriented Middleware (MOM), task distribution in microservices, and implementing reliable communication patterns between agents.

Pipes are a unidirectional IPC channel, typically used for communication between a parent and child process. Data written to the write end is read from the read end. Named Pipes (or FIFOs) exist as a file in the filesystem, allowing unrelated processes to communicate.

• Anonymous Pipes: Created with the pipe() system call, exist only for the lifetime of the processes, and are usually used for related processes.
• Named Pipes (FIFOs): Created with mkfifo, have a persistent presence in the filesystem, enabling communication between any processes that know the pipe's name.
• Use Case: Pipes are classic in Unix/Linux for connecting the standard output of one command to the standard input of another (ls | grep). Named pipes are used for simple, persistent inter-process data streams.

Sockets are a generalized network IPC interface that can facilitate communication between processes on the same machine (Unix Domain Sockets) or across a network (Network Sockets using TCP/UDP). They provide a full-duplex, bidirectional communication channel.

• Unix Domain Sockets: Extremely efficient for local IPC, using the filesystem as a namespace for addressing.
• Network Sockets: Enable distributed systems. Protocols like TCP provide reliable, ordered delivery, while UDP offers connectionless, best-effort datagrams.
• Use Case: The backbone of networked applications, client-server architectures, and is the underlying transport for higher-level protocols like gRPC and WebSocket connections.

Remote Procedure Call (RPC) is a protocol that allows a program to execute a function or procedure on another address space (another process or machine) as if it were a local call. It abstracts the complexities of network communication.

• Mechanism: The client calls a local stub function, which marshals parameters into a message, sends it to the server, waits for the response, unmarshals it, and returns the result.
• Modern Implementations: Frameworks like gRPC (using HTTP/2 and Protocol Buffers) and JSON-RPC provide structured, efficient RPC.
• Use Case: Fundamental for building distributed systems, microservices, and client-agent interactions where a synchronous request-response semantic is required.

Semaphores and Mutexes are synchronization primitives, not data-transfer IPC mechanisms per se, but are critical for safe IPC. They control access to shared resources (like shared memory or a queue) to prevent race conditions.

• Semaphore: A counter used to control access to a pool of resources. Operations are wait() (decrement) and signal() (increment).
• Mutex (Mutual Exclusion): A binary lock that ensures only one thread or process can enter a critical section of code at a time.
• IPC Role: When using Shared Memory or any shared state, these primitives are mandatory to ensure data integrity and coordinated access between concurrent processes.

A Message Queue is a fundamental IPC buffer that stores messages in a First-In-First-Out (FIFO) order, enabling asynchronous and decoupled communication. The sending process enqueues a message and continues execution without waiting, while the receiving process dequeues it when ready. This provides loose coupling and load leveling. Common implementations include POSIX message queues and broker-based systems like RabbitMQ.

Shared Memory is the fastest IPC mechanism, allowing multiple processes to access a common region of RAM directly. It bypasses kernel intervention required by other methods but requires explicit synchronization primitives (like semaphores or mutexes) to prevent race conditions and ensure data consistency. It is ideal for high-performance, high-volume data exchange where processes are on the same physical machine.

A Remote Procedure Call (RPC) is a protocol that allows a program to execute a procedure (subroutine) in another address space (often on a different machine) as if it were a local function call. It abstracts network communication into a familiar programming paradigm. Key implementations include:

• gRPC: Uses HTTP/2 and Protocol Buffers for high-performance, type-safe contracts.
• JSON-RPC: A lightweight protocol using JSON for serialization. RPC typically follows a synchronous request-response pattern.

Publish-Subscribe (Pub/Sub) is a messaging pattern where senders (publishers) categorize messages into topics without knowing the specific receivers. Receivers (subscribers) express interest in one or more topics and receive relevant messages asynchronously. This pattern enables broadcast communication and dynamic, many-to-many relationships. It is a core pattern in event-driven architectures and is often implemented by message brokers like Apache Kafka or Redis Pub/Sub.

Pipes are a unidirectional IPC channel, typically used for communication between a parent and child process. Data written to the write end is read from the read end. Named Pipes (or FIFOs) exist as a file in the filesystem, allowing unrelated processes to communicate. They provide a simple byte-stream interface but are generally less flexible than message queues or sockets for complex coordination.

Sockets provide a bidirectional communication endpoint, most commonly used for network communication between processes on different machines (Internet sockets), but also for communication on the same host (Unix domain sockets). They support multiple protocols (e.g., TCP for reliable streams, UDP for datagrams) and are the foundation for most networked IPC. WebSockets build upon this model to provide full-duplex communication over a single, long-lived TCP connection.