Compute Express Link (CXL)

CXL defines a hardware-managed cache coherence protocol that allows the CPU and connected devices (like NPUs, GPUs, or memory buffers) to share a unified, consistent view of memory. This eliminates the need for software to manually flush caches, a major source of latency and complexity in traditional accelerator interconnects like PCIe.

• Key Mechanism: Uses a snoop filter to track which agents cache which memory lines, minimizing coherence traffic.
• Benefit: Enables fine-grained data sharing between CPU and device, allowing them to work on the same dataset without data corruption or explicit data movement commands.

CXL supports load/store memory semantics, allowing a device to access host memory or other device memory using standard CPU-like read/write operations. This enables advanced use cases like memory pooling and memory expansion.

• Memory Pooling: A CXL-attached memory device can be dynamically allocated to different hosts in a data center, improving utilization.
• Memory Expansion: Systems can augment their DRAM capacity with lower-cost, higher-density CXL-attached memory (CXL.mem), appearing as part of the system's physical address space.
• Example: A server with 512GB of local DRAM could add a 2TB CXL.mem module, presenting a total of 2.5TB of usable memory to the OS.

CXL operates over a physical PCI Express (PCIe) 5.0+ electrical interface but uses its own logical protocol layers. The stack is composed of three optional protocol 'flavors' that can be dynamically negotiated:

• CXL.io: Essentially PCIe, providing foundational I/O operations for device discovery, configuration, and interrupts.
• CXL.cache: Allows a device (e.g., an NPU) to cache host memory, requiring the coherence protocol.
• CXL.mem: Allows the host CPU to access device-attached memory (like an expansion module) or a device to access host memory.

A device like an NPU typically uses CXL.io + CXL.cache, while a memory expander uses CXL.io + CXL.mem.

Built on the PCIe 5.0/6.0 physical layer, CXL is designed for ultra-low latency and high bidirectional bandwidth, critical for accelerator workloads where data movement overhead can dominate execution time.

• Latency: Aims for latency on the order of ~100 nanoseconds for coherent accesses, significantly lower than software-managed transfers over standard PCIe.
• Bandwidth: Leverages PCIe's lanes. A x16 link using PCIe 5.0 provides ~64 GB/s per direction (128 GB/s bidirectional). PCIe 6.0 doubles this.
• Use Case: An NPU can fetch model weights or intermediate activation tensors from host memory with minimal delay, keeping its compute units saturated.

CXL is a key enabler for disaggregated and composable infrastructure. It allows compute, memory, and storage resources to be physically separated in a data center rack and then logically composed into virtual servers on-demand.

• Disaggregation: CPUs, pools of CXL-attached memory, and banks of NPUs can be separate hardware modules.
• Dynamic Composition: A workload requiring massive memory and an NPU can be assigned a CPU socket, a slice of memory from a CXL pool, and a dedicated NPU, all connected via the CXL fabric.
• Benefit: Dramatically improves hardware utilization and enables more flexible, efficient data center resource management.

CXL is developed and maintained by the CXL Consortium, an industry group with broad membership from CPU vendors (Intel, AMD, ARM), device makers, and cloud providers. This open standard is crucial for interoperability and avoiding vendor lock-in.

• Versions: CXL 1.0/1.1 established the base protocol. CXL 2.0 added switching and memory pooling. CXL 3.0 introduced fabric capabilities and peer-to-peer communication between devices.
• Ecosystem Impact: Drives a common software model (e.g., OS and hypervisor support) and hardware compliance, allowing system builders to integrate components from different vendors confidently.
• Contrast: Unlike proprietary interconnects, CXL's openness fosters competition and innovation across the compute, memory, and accelerator landscape.

Cache coherence is a fundamental property of multi-processor systems that ensures all processor caches have a consistent view of shared memory. Without it, different processors could read stale or conflicting values for the same memory location, leading to incorrect program execution. CXL builds upon and extends cache coherence protocols (like MESI) across the interconnect to accelerators and memory devices, enabling them to participate in a unified, coherent memory domain with the host CPU. This is a core feature of CXL.mem and CXL.cache protocols.

Non-Uniform Memory Access (NUMA) is a multiprocessor memory architecture where memory access time depends on the memory location relative to the processor. Local memory is faster to access than memory attached to another processor socket. CXL enables a form of heterogeneous NUMA by allowing memory to be attached via the CXL link. This memory appears in the system's physical address space but has higher latency than local DDR memory. System software and applications must be aware of these latency/bandwidth tiers to optimize data placement, a concept known as CXL Memory Tiering.

High Bandwidth Memory (HBM) is a 3D-stacked DRAM technology providing extremely high bandwidth in a compact form factor, directly connected to a processor or accelerator die via a wide, ultra-short interface (like a silicon interposer). It represents the fastest tier in a system's memory hierarchy. CXL operates at a different level: it is an interconnect for attaching external memory and devices. An NPU might use HBM for its ultra-fast local workspace, while using CXL to efficiently access much larger pools of host or pooled memory, or to enable cache-coherent sharing of data with the CPU.

Peripheral Component Interconnect Express (PCIe) is the ubiquitous, high-speed serial expansion bus standard that CXL is built upon. CXL uses the physical and electrical layers of PCIe 5.0+ (the wires, connectors, and basic signaling) but adds its own logical protocol layer on top. This allows CXL devices to plug into standard PCIe slots. The key difference is semantics: PCIe is primarily a load/store I/O interface, while CXL adds cache coherence and memory semantics, transforming it from a mere I/O bus into a memory-centric interconnect for accelerators and memory expansion.

Direct Memory Access (DMA) is a capability that allows a hardware subsystem (like an NPU or network card) to access host system memory independently of the CPU, offloading data transfer tasks. Traditional DMA requires software to manage data coherence explicitly. CXL enhances this model: a CXL-attached accelerator using the CXL.cache protocol can perform DMA-like accesses that are automatically cache-coherent with the host CPU. This eliminates the need for software to manually flush caches, simplifying programming and improving performance for fine-grained data sharing between the CPU and accelerator.

Memory pooling is a resource utilization paradigm where a large block of memory (like a CXL memory expander) is shared dynamically among multiple compute hosts (servers) over a network or fabric. CXL is a key enabling technology for memory pooling. A CXL Switch allows several host CPUs to connect to a shared set of CXL Type 3 memory devices. This allows memory capacity to be decoupled from individual servers, enabling more efficient use (reducing stranded memory) and allowing servers to dynamically scale memory up or down based on workload demands, similar to cloud computing models.