Shared Memory Architecture

The defining feature of a shared memory architecture is a single, logical address space that all participating agents can directly read from and write to. This eliminates the need for explicit message-passing protocols for data transfer, as agents communicate by modifying shared variables.

• Direct Access: Agents use standard memory load/store operations.
• Abstraction: Presents a simplified programming model, akin to multi-threaded programming on a single machine.
• Challenge: Requires sophisticated concurrency control (e.g., locks, semaphores) to prevent race conditions and ensure data integrity when multiple agents access the same location.

By enabling agents to read the latest state written by another agent directly from memory, this architecture minimizes communication latency. The speed is bounded primarily by memory access times and interconnect bandwidth, not by network protocol overhead.

• Performance: Critical for tightly-coupled, collaborative tasks where agents must react to each other's state changes in microseconds or milliseconds.
• Contrast: Compared to distributed memory (message-passing) systems, shared memory avoids serialization/deserialization and network stack delays for shared data.
• Trade-off: This low latency often comes with constraints on physical scalability, as maintaining a coherent view across many nodes becomes challenging.

A consistency model is a formal contract between the memory system and the agents, specifying the guarantees about the order in which memory operations become visible. It answers the question: "If Agent A writes to location X, when will Agent B see that write?"

• Strong Consistency: Guarantees that any read returns the value of the most recent write. Simplifies reasoning but impacts performance.
• Weaker Models: Architectures may implement models like sequential consistency or release consistency to improve performance by relaxing strict ordering, requiring explicit synchronization operations from programmers.
• Criticality: The chosen model directly impacts the complexity of developing correct concurrent software for the system.

Shared access necessitates mechanisms to coordinate agents and prevent data races (non-deterministic outcomes from unsynchronized concurrent accesses). The architecture must provide hardware or software synchronization primitives.

• Atomic Operations: Instructions like compare-and-swap (CAS) or fetch-and-add that are indivisible, used to build locks and lock-free data structures.
• Locks & Mutexes: Mechanisms that grant exclusive access to a critical section of code or data.
• Semaphores & Barriers: Higher-level constructs for controlling access to a pool of resources or synchronizing agent progress.
• Overhead: Excessive synchronization can serialize execution and become a performance bottleneck, negating the benefits of parallelism.

These are two distinct but related concepts in shared memory systems:

• Cache Coherence: A property of a hardware-based shared memory system (e.g., a multi-core CPU). It guarantees that all caches have a consistent view of a single memory location. If one core writes to address A, all other cores' caches are invalidated or updated. This is typically transparent to software.
• Memory Consistency: As defined above, this is the programmer-visible guarantee about the order of reads and writes to different memory locations. It is defined by the consistency model.
• Distinction: Coherence deals with the technical replication of a single variable; consistency deals with the logical ordering of operations on multiple variables.

While ideal for small-to-medium scale coordination, pure shared memory faces fundamental scalability limits as the number of agents or nodes increases.

• Contention: Frequent writes to a shared location create a hotspot, causing agents to stall waiting for access.
• Coherence Overhead: In distributed shared memory (DSM) systems, maintaining cache coherence across a network generates significant traffic and latency (false sharing is a common problem).
• Physical Limits: Bus-based interconnects saturate. Scalable systems often use hierarchical or directory-based coherence protocols, which add complexity.
• Architectural Hybrids: Many large-scale systems use a hybrid approach, combining shared memory within a node with message-passing between nodes.

A formal specification that defines the ordering guarantees and visibility of memory operations (reads and writes) across multiple agents or processors in a concurrent system. It answers the question: "What write value will a read operation see?" Different models offer trade-offs between performance and programmer simplicity.

• Strong Consistency: Guarantees any read returns the most recent write. Simplifies reasoning but is high-latency.
• Eventual Consistency: Guarantees that if no new updates are made, all reads will eventually return the last value. Enables high availability.
• Causal Consistency: Preserves the order of causally related operations, allowing concurrent operations to be seen in different orders.

A data structure designed for distributed systems that can be updated concurrently by multiple agents without coordination, and whose state can always be merged deterministically. CRDTs are ideal for implementing shared memory in eventually consistent systems because they guarantee convergence.

• Operation-based CRDTs: Agents broadcast their update operations, which are applied in a commutative order.
• State-based CRDTs: Agents exchange their full state, merging them using a commutative, associative, and idempotent function.
• Common Examples: G-Counters (grow-only), PN-Counters (positive/negative), OR-Sets (observed-remove sets) for collaborative editing.

The methodology for copying and maintaining data across multiple nodes in a distributed system to improve availability, fault tolerance, and read performance. The strategy defines how writes are propagated and synchronized.

• Leader-Follower (Primary-Secondary): A single leader handles all writes, synchronously or asynchronously replicating to followers. Simple but a write bottleneck.
• Multi-Leader: Multiple nodes accept writes, requiring conflict resolution (e.g., using CRDTs or last-write-wins). Increases write throughput but adds complexity.
• Leaderless (Dynamo-style): Writes are sent to a quorum of nodes; reads also query a quorum, reconciling versions if needed.

A coordination service that provides mutually exclusive access to a shared resource (like a memory segment or data record) across multiple nodes in a distributed system. It prevents race conditions by ensuring only one agent can hold a lock on a resource at a time.

• Mechanism: Implements protocols like Paxos or Raft to maintain consensus on lock ownership.
• Leases: Locks are often granted as time-bound leases to prevent deadlock if a client fails.
• Use Case: Coordinating access to a non-CRDT resource that requires strong consistency, such as a configuration file or a unique ID generator.

A sequence of memory operations (reads and writes) that are executed as a single, atomic unit, ensuring the system transitions from one consistent state to another. Transactions provide the ACID guarantees (Atomicity, Consistency, Isolation, Durability) for shared memory operations.

• Atomicity: All operations in the transaction succeed or none do.
• Isolation: Concurrent transactions do not interfere with each other (implemented via locking or Multi-Version Concurrency Control).
• Distributed Transactions: Require protocols like Two-Phase Commit (2PC) to coordinate atomic commits across multiple nodes, which is complex and can block on failures.

A messaging middleware pattern that facilitates decoupled communication between agents by allowing them to publish events to and subscribe to topics. This is an asynchronous alternative to direct shared memory access for coordination and state change notification.

• Publish-Subscribe (Pub/Sub): Senders categorize messages into topics without knowledge of receivers. Subscribers receive all messages for topics they've subscribed to.
• Event Sourcing: The shared state is derived from an immutable log of all events (state changes). Agents rebuild state by replaying the log.
• Use Case: Broadcasting a state change (e.g., "item inventory updated") to all interested agents without them polling shared memory.