Memory Consistency Model

Sequential Consistency is the most intuitive and strongest model, providing a simple mental model for programmers. It guarantees that the result of any execution is the same as if the operations of all threads were interleaved in some sequential order, and the operations of each individual thread appear in this sequence in the order specified by its program.

• Key Guarantee: All threads observe a single, global, sequential order of all memory operations.
• Performance Impact: Enforcing this global order requires significant synchronization (e.g., frequent memory barriers), which can limit hardware optimizations like write buffering and out-of-order execution, potentially reducing throughput.
• Use Case: Often used as a reference model for reasoning about correctness, but rarely implemented in its pure form in high-performance hardware due to its strict constraints.

Total Store Order is the model implemented by x86 and SPARC architectures. It relaxes Sequential Consistency in one critical aspect: it allows a thread's store operations to be buffered in a write queue before becoming visible to other threads, while maintaining program order for all other operation pairs.

• Key Relaxation: A load may bypass a prior store to a different address. This creates the potential for the load-load reordering anomaly, where two threads see writes in different orders.
• Hardware Motivation: The write buffer is essential for hiding the latency of store operations, dramatically improving performance.
• Synchronization: The model still requires explicit memory fences (like MFENCE on x86) to enforce ordering when needed for correctness, such as in lock implementations.

Release/Acquire is a programmer-centric model that provides synchronization guarantees only at specific points, rather than on all memory operations. It is the foundation for C++11, Java, and Rust memory models.

• Synchronizing Operations: Ordering is enforced between pairs of operations:
  • A release operation (e.g., store-release, unlock) ensures all prior memory accesses in this thread are visible to other threads.
  • A subsequent acquire operation (e.g., load-acquire, lock) on the same atomic variable by another thread guarantees it sees all writes from the thread that performed the release.
• Performance Benefit: Non-synchronizing loads and stores (ordinary accesses) can be freely reordered by the hardware and compiler, enabling aggressive optimization.
• Data-Race-Free: Correct programs use these synchronization operations to prevent data races, creating a "happens-before" relationship that guarantees sequential consistency for correctly synchronized code.

Weak Consistency (or Relaxed memory order) provides minimal guarantees, allowing most memory operations to be reordered. It offers the highest potential performance but places the greatest burden on the programmer to enforce ordering explicitly.

• Key Guarantee: Only data dependencies and explicit memory barriers enforce order. Loads and stores to different addresses can be observed in any order by other threads.
• Hardware Use: Common in ARM (AArch64) and PowerPC architectures, and critically, in many NPU and GPU programming models (e.g., CUDA, OpenCL). These accelerators rely on massive parallelism and deep memory hierarchies where strict ordering is prohibitively expensive.
• Programming Model: Correctness requires careful insertion of barriers (e.g., __threadfence() in CUDA, dmb instruction on ARM) to ensure visibility of results before they are consumed by other threads or workgroups.

This is not a hardware model per se, but a fundamental theorem linking weak hardware models to strong programmer guarantees. It states that if a program is written to be data-race-free (using proper synchronization like locks or atomics), then the hardware will provide the illusion of Sequential Consistency to that program.

• Compiler & Hardware Contract: This theorem allows compilers to perform aggressive optimizations and hardware to implement weak memory models, safe in the knowledge that correctly synchronized programs will still behave as the programmer intended.
• Foundation for High-Level Languages: This principle underpins the memory models of Java, C++, and Rust. The language guarantees SC for data-race-free programs, while allowing implementations to map to the weaker, more efficient models of underlying hardware like ARM or NPUs.

Accelerator memory models are designed for hierarchical, massively parallel execution. They explicitly expose memory scopes and require manual management.

• Memory Scopes: Operations can be ordered within a thread, a thread block (cooperative group), or the device (all threads). Weaker ordering is the default; stronger scopes require explicit fences.
• Hierarchical Synchronization:
  • __syncthreads() ensures visibility within a thread block.
  • Device-wide fences (__threadfence_system()) are expensive and used sparingly.
• Relaxed Atomics: NPU/GPU programming often uses relaxed atomics for performance, where ordering guarantees are minimal, and the programmer must use fences to establish necessary visibility. This model maximizes throughput by aligning with the hardware's non-coherent cache hierarchy and SIMT execution.