CXL is an open standard interconnect that maintains cache coherency between the CPU and attached devices, allowing accelerators and memory expanders to share a unified, consistent view of system memory. Built upon the physical and electrical layer of PCI Express (PCIe) 5.0/6.0, it introduces three distinct protocols: CXL.io for discovery and configuration, CXL.cache for coherent caching, and CXL.mem for direct memory access, enabling true resource disaggregation in composable server architectures.
Glossary
CXL
An open industry-standard cache-coherent interconnect that enables high-bandwidth, low-latency communication between CPUs, GPUs, memory, and other accelerators, enabling resource disaggregation.
INTERCONNECT STANDARD
What is CXL?
Compute Express Link (CXL) is an open industry-standard cache-coherent interconnect built on PCIe physical infrastructure, enabling high-bandwidth, low-latency communication between CPUs, GPUs, memory, and accelerators.
The primary strategic value of CXL lies in breaking the fixed ratio of compute to memory. It allows memory pooling, where multiple hosts can dynamically share a common pool of memory, and memory tiering, where cheaper, higher-latency memory is seamlessly integrated with local DRAM. For AI infrastructure, this means GPU clusters can access significantly larger memory pools beyond their local HBM3e capacity, reducing the need for costly data movement and enabling the training of larger models on existing silicon.
COMPUTE EXPRESS LINK
Key Features of CXL
CXL is an open industry-standard cache-coherent interconnect built on PCIe 5.0/6.0 that enables high-bandwidth, low-latency communication between CPUs, GPUs, memory, and accelerators. Its three protocols (CXL.io, CXL.cache, CXL.mem) fundamentally enable resource disaggregation and composable infrastructure.
01
Type 1: Cache-Coherent Accelerators (CXL.cache)
Designed for devices like SmartNICs and DPUs that have their own cache but no local memory. CXL.cache allows the device to cache host memory coherently using the CXL 68-byte flit format.
- Enables accelerators to participate in the CPU's cache coherence domain
- Uses MESI-based coherence protocols to maintain consistency
- Ideal for inline encryption, compression, and network processing offloads
- Eliminates costly driver-mediated data copies between host and device
02
Type 2: Accelerators with Memory (CXL.cache + CXL.mem)
Targets GPUs, FPGAs, and ASICs that possess local high-bandwidth memory (HBM/GDDR) and can optionally expose it to the host. This type supports both caching host memory and exposing device-attached memory to the CPU's address space.
- Enables unified virtual addressing between host CPU and accelerator
- Allows the host to directly read/write device-attached HBM via load/store semantics
- Critical for heterogeneous computing where CPU and accelerator share data structures
- Reduces programming complexity by eliminating explicit memory copy APIs
03
Type 3: Memory Expanders (CXL.mem)
The most commercially mature CXL type. Type 3 devices are memory buffers that expose DRAM or persistent memory to the host via CXL.mem, enabling memory tiering and capacity expansion beyond DIMM slot limitations.
- Supports DDR4, DDR5, and persistent memory media behind the CXL controller
- Enables memory pooling where multiple hosts share a single CXL memory appliance
- Latency penalty of ~170-250ns compared to direct-attached DDR5 (~80-100ns)
- Samsung, Micron, and SK hynix have released CXL 2.0 memory modules with up to 512GB per device
04
CXL 2.0: Switching and Pooling
CXL 2.0 introduced CXL switches and memory pooling, moving beyond point-to-point topologies. A CXL switch enables multiple hosts to connect to multiple Type 3 devices, dynamically allocating memory resources.
- Supports MLD (Multi-Logical Device) where a single physical device is partitioned into multiple logical devices assigned to different hosts
- Enables memory pooling where a rack-scale pool of CXL memory can be dynamically assigned to servers based on workload demand
- Uses PBR (Port-Based Routing) for efficient switching fabric traversal
- Foundation for true composable disaggregated infrastructure (CDI)
05
CXL 3.0/3.1: Fabric Capabilities and Peer-to-Peer
CXL 3.0 (PCIe 6.0 base) doubled bandwidth to 64 GT/s and introduced multi-level switching and peer-to-peer DMA between devices without host intervention. CXL 3.1 added TSP (Trusted Security Protocol) for confidential computing.
- P2P DMA: GPU can directly access CXL-attached memory without CPU involvement
- Global Fabric Attached Memory (GFAM) with shared memory regions across multiple hosts
- Back-invalidation for improved cache coherence in multi-headed device topologies
- Enhanced IDE (Integrity and Data Encryption) for secure data-in-flight protection
06
CXL in AI Infrastructure: GPU Memory Expansion
CXL is emerging as a critical technology for GPU memory disaggregation in AI clusters. By attaching CXL Type 3 memory expanders to GPU hosts, systems can address the memory capacity bottleneck in large model training and inference.
- Panmnesia and Samsung have demonstrated CXL-based GPU memory expansion for LLM inference
- Enables tiered memory where hot tensors reside in HBM and cold data spills to CXL-attached DDR5
- Reduces the need for costly model parallelism strategies by expanding per-GPU addressable memory
- CXL over Optics research aims to extend memory pooling across rack-scale distances
CXL INTERCONNECT
Frequently Asked Questions
Common questions about the Compute Express Link (CXL) standard, its role in disaggregated computing, and its impact on AI infrastructure architectures.
Compute Express Link (CXL) is an open industry-standard cache-coherent interconnect built on the PCI Express (PCIe) physical layer that enables high-bandwidth, low-latency communication between CPUs, GPUs, memory, and other accelerators. It works by extending the PCIe protocol with three distinct sub-protocols: CXL.io (for discovery, configuration, and I/O), CXL.cache (allowing devices to cache host memory), and CXL.mem (allowing the host to access device-attached memory using load/store semantics). This cache-coherent fabric allows accelerators like GPUs and FPGAs to share a unified memory space with the host CPU, eliminating the need for costly data copies and enabling true resource disaggregation in composable infrastructure.
INTERCONNECT COMPARISON
CXL vs. NVLink vs. InfiniBand
A technical comparison of three high-performance interconnects used in modern AI infrastructure, contrasting their primary use cases, topologies, and architectural roles.
| Feature | CXL | NVLink | InfiniBand |
|---|---|---|---|
Primary Role | Cache-coherent memory & resource disaggregation | Intra-node GPU-to-GPU high-bandwidth link | Inter-node high-performance cluster networking |
Coherency Model | Cache-coherent (CXL.cache, CXL.mem) | Cache-coherent within a single address space | Non-coherent (RDMA-based data transfer) |
Typical Topology | Switched, tree, or direct-attach | Direct attach or NVSwitch mesh | Switched fabric (Fat-Tree, Dragonfly+) |
Max Bandwidth (per link) | 64 GB/s (CXL 3.0 x16) | 900 GB/s (NVLink 4.0, bidirectional) | 400 GB/s (NDR400, bidirectional) |
Latency | < 100 ns (host-to-device) | < 10 ns (GPU-to-GPU) | ~1 µs (switch port-to-port) |
Protocol Layer | PCIe 5.0/6.0 physical layer | Proprietary NVIDIA physical layer | IBTA physical layer (1-4 lanes) |
CPU Bypass | |||
Memory Pooling Support |
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.
Enterprise searchRAGPermissions
Read more
Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.
AI agentsWorkflow automationGovernance
Read more
Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
AI integrationDecision supportModel routing
Read more
About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
LinkedIn
Limited slotsGet a Free AI Consultation
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us