Compute Express Link (CXL)

The foundational protocol layer, CXL.io, is essentially PCIe 5.0+ with CXL-specific enhancements. It provides the basic I/O semantics for device discovery, configuration, and initialization, ensuring full backward compatibility with the vast PCIe ecosystem. This layer handles:

• Enumeration and configuration of CXL devices.
• Standard DMA (Direct Memory Access) operations.
• Legacy interrupt and error handling. It serves as the essential transport for the higher-level CXL.mem and CXL.cache protocols.

The CXL.cache protocol enables a device (like a GPU or FPGA) to cache host CPU memory coherently. It defines a set of transactions that allow the device to:

• Request and snoop cache lines from the host's memory.
• Maintain coherency using a directory-based or snoop-based model.
• Reduce latency for accelerator access to frequently used data. This protocol is critical for workloads where the accelerator needs low-latency, coherent access to data structures managed by the CPU, avoiding costly software-managed coherence overhead.

The CXL.mem protocol allows the host processor to access device-attached memory (e.g., pooled DRAM, persistent memory) using load/store instructions, making it appear as part of the system's unified memory address space. Key functions include:

• Host-initiated reads and writes to device memory.
• Support for memory pooling and expansion.
• Atomic operations for synchronization. This enables efficient memory capacity scaling and is fundamental for memory-centric architectures and composable infrastructure.

Operating above the physical layer (PCIe), these layers manage reliable packet delivery and transaction semantics:

• Transaction Layer: Generates and processes Transaction Layer Packets (TLPs) for read/write requests and completions. It is shared across CXL.io, .cache, and .mem, with specific Transaction IDs and attributes for each protocol.
• Link Layer: Ensures data integrity via CRC (Cyclic Redundancy Check) and manages flow control and link state through a shared credit system. It handles packet acknowledgment and retry mechanisms for reliable transmission.

CXL 1.x/2.x uses the PCIe 5.0 physical layer, and CXL 3.0 uses PCIe 6.0. This includes:

• Electrical specifications (signaling, equalization).
• Lane configuration (x16, x8, etc.).
• Initialization and training sequence. The shared physical layer is why a CXL device can operate in legacy PCIe mode during boot. The key differentiator is the Flex Bus interface, which dynamically negotiates whether the link operates in PCIe or CXL mode after initialization.

CXL defines device types based on which protocols they implement, dictating their functionality:

• Type 1 Devices (Accelerators): Use CXL.io + CXL.cache. Example: Smart NICs or FPGAs that cache host memory.
• Type 2 Devices (GPUs/AI Accelerators): Use CXL.io + CXL.cache + CXL.mem. They have their own memory but can also cache host memory and offer it to the host.
• Type 3 Devices (Memory Buffers): Use CXL.io + CXL.mem. Example: Memory expanders or persistent memory devices that provide host-accessible memory but do no compute.

CXL's defining feature is its maintenance of memory coherency between the host CPU and connected devices (e.g., GPUs, FPGAs, Smart NICs). This allows accelerators to share a unified memory space with the CPU, eliminating the need for explicit, software-managed data copies. Key protocols include:

• CXL.io: Provides foundational PCIe-like I/O semantics.
• CXL.cache: Enables devices to cache host memory.
• CXL.mem: Allows the host CPU to access device-attached memory. This coherency is critical for AI workloads where data must move seamlessly between CPU control logic and accelerator compute units.

CXL enables memory disaggregation, allowing pooled memory resources to be dynamically allocated to multiple hosts or accelerators. This is a game-changer for memory-intensive AI models. A CXL-attached memory buffer or pooling device appears as byte-addressable memory to the host system.

• Use Case: Training extremely large language models (LLMs) that exceed the physical DRAM capacity of a single server.
• Benefit: Provides a cost-effective, high-bandwidth alternative to purchasing servers with maximum installed DRAM, improving overall cluster utilization and enabling larger batch sizes or longer context windows.

CXL is the preferred attach point for next-generation AI accelerators (beyond GPUs), such as Data Processing Units (DPUs), intelligence Processing Units (IPUs), and specialized neural network accelerators. It provides a unified, coherent interface that simplifies programming.

• Comparison to PCIe: While built on the physical PCIe layer, CXL adds the crucial cache coherency protocol, moving beyond a simple I/O bus to a true memory-semantic fabric.
• Impact: Reduces software complexity, lowers latency for fine-grained workloads, and enables tighter coupling of CPUs with specialized AI chips for tasks like preprocessing, inference, or custom operators.

CXL is a foundational technology for composable disaggregated infrastructure (CDI). In AI data centers, resources like GPU clusters, memory pools, and storage can be physically disaggregated and then logically composed into virtual servers on-demand via a CXL fabric.

• Dynamic Resource Allocation: An AI training job can be provisioned with 8 GPUs and 2TB of pooled CXL memory from a shared resource pool.
• Efficiency: Dramatically improves hardware utilization by breaking the rigid, fixed configuration of traditional servers, allowing infrastructure to elastically match the needs of varying AI workloads.

CXL is managed by the CXL Consortium. Key versions include:

• CXL 1.1/2.0: Established the core protocols (io, cache, mem).
• CXL 3.0/3.1: Introduced fabric capabilities, memory pooling, multi-headed devices, and peer-to-peer communication between devices, enabling true disaggregation. It also doubled the per-lane data rate. The standard leverages the PCI Express physical and electrical interface, ensuring backward compatibility while adding the essential coherency and memory semantics on top.

CXL interacts with several key AI/hardware concepts:

• NVLink (NVIDIA): A proprietary, ultra-high-bandwidth GPU-to-GPU interconnect. CXL is an open standard for CPU-to-device/device-to-device.
• Unified Memory (e.g., CUDA UVA): A software abstraction for a unified address space. CXL provides the hardware-level coherence to make such abstractions more efficient and scalable.
• Gen-Z, CCIX: Earlier cache-coherent interconnect proposals largely superseded by the industry convergence on CXL.
• UCIe (Universal Chiplet Interconnect Express): A die-to-die interconnect standard. CXL or PCIe is often used as the protocol layer running over a UCIe physical layer for chiplet-based accelerators.

Cache Coherence is the property that ensures multiple processing units (CPUs, accelerators) in a shared-memory system have a consistent view of data. CXL's CXL.cache protocol is the mechanism that extends this coherence domain.

• Domain Extension: Allows a device (e.g., a CXL-attached GPU or FPGA) to cache host memory, with the CPU managing coherence.
• Snooping & Directory-Based: CXL.cache uses a snoop filter model, where the host manages a directory of cached lines to minimize snoop traffic.
• Key Benefit: Enables accelerators to work on memory directly, avoiding costly software-managed copies and synchronization.

CCIX (Cache Coherent Interconnect for Accelerators) and OpenCAPI (Open Coherent Accelerator Processor Interface) were competing, CPU-architecture-specific standards for cache-coherent accelerator attachment that preceded CXL's industry consolidation.

• CCIX: Championed by Arm, AMD, and Xilinx, it provided cache coherency over a PCIe-like link.
• OpenCAPI: An open interface championed by IBM and partners, offering very low latency and high bandwidth.
• Industry Convergence: With Intel, AMD, and Arm all backing CXL 3.0+, it has become the unified, dominant standard, subsuming the need for these separate interconnects.

CXL Memory Pooling is a use case enabled by CXL 3.0+ where a block of memory attached via CXL (e.g., on a CXL Type 3 memory expander) can be dynamically allocated and shared among multiple host processors in a system.

• Dynamic Capacity: Memory can be assigned to hosts on-demand, improving utilization versus statically attached DIMMs.
• Fabric-Based: CXL 3.0 introduces switching capabilities, allowing memory devices to be connected to multiple hosts over a CXL fabric.
• Use Case: Ideal for cloud and composable infrastructure, allowing memory to be provisioned like compute and storage resources.

The CXL specification defines three primary device types based on which protocol layers they implement, determining their functionality and use case.

• Type 1 (CXL.cache): Accelerators (e.g., Smart NICs, FPGAs) that cache host memory but have little or no local memory. They use CXL.io and CXL.cache.
• Type 2 (CXL.cache + CXL.mem): GPUs & ASICs with their own attached memory (GDDR/HBM) that also cache and access host memory. They use all three protocols.
• Type 3 (CXL.mem): Memory Expanders & Pooling Devices (e.g., DDR or persistent memory attached via CXL). They use CXL.io and CXL.mem to provide additional memory capacity to the host.