Direct Liquid Cooling

A cold plate is a thermally conductive metal block (usually copper or aluminum) mounted directly onto a chip's lid. Coolant flows through internal micro-channels or fins, absorbing heat via conduction. This is the most common DLC method for GPUs.

• Micro-channel design: Channels as narrow as 50-100 microns maximize surface area for heat transfer
• Thermal interface material (TIM): A high-conductivity paste or indium foil fills microscopic air gaps between the chip and cold plate
• Single-phase vs. two-phase: Single-phase systems keep coolant liquid throughout; two-phase allows coolant to boil, leveraging latent heat of vaporization for higher heat flux removal

The CDU is the heart of a DLC system, managing coolant flow, pressure, temperature, and filtration between the facility water loop and the secondary loop that feeds the cold plates.

• Heat exchanger: Transfers thermal energy from the secondary (tech) loop to the primary (facility) loop without mixing fluids
• Redundant pumps: Variable-speed pumps maintain precise flow rates; N+1 redundancy ensures uptime during pump failure
• Filtration and deionization: Maintains coolant purity to prevent corrosion, biological growth, and electrical conductivity in the secondary loop

Unlike cold plates, immersion cooling submerges entire servers or components in a thermally conductive, electrically non-conductive dielectric fluid. This eliminates the need for fans and allows for uniform cooling of all components.

• Single-phase immersion: Hardware sits in a sealed tank of dielectric fluid; pumps circulate fluid to an external heat exchanger
• Two-phase immersion: Fluid boils on contact with hot components; vapor rises, condenses on a cooled coil, and drips back down
• Material compatibility: All components—cables, labels, adhesives—must be certified for long-term dielectric fluid exposure to prevent leaching or degradation

A critical safety subsystem in any DLC deployment. Leak detection uses physical sensors and software monitoring to identify coolant breaches before they damage hardware.

• Rope sensors: Conductive cables placed along potential leak paths; resistance changes when wetted trigger alerts
• Pressure decay testing: Automated pressure monitoring detects micro-leaks by measuring pressure drop over time in isolated loop segments
• Drip trays and containment: Physical barriers channel leaked fluid away from electronics into collection reservoirs, preventing cascading failures

DLC systems must interface with the building's facility water loop, which rejects heat to the outside environment via cooling towers, dry coolers, or chillers.

• Warm water cooling: Modern DLC operates with supply temperatures up to 40-45°C (104-113°F), enabling free cooling via ambient air in most climates without mechanical chillers
• Water quality management: Corrosion inhibitors, biocides, and filtration prevent scaling and biological fouling in the primary loop
• Heat reuse potential: High-grade waste heat (40-60°C) can be repurposed for district heating, greenhouses, or absorption chillers, improving overall energy efficiency

The secondary loop coolant is typically deionized water with corrosion inhibitors and biocides, or a propylene glycol/water mix for freeze protection. Maintaining proper chemistry is essential for long-term reliability.

• Conductivity monitoring: Maintains electrical resistivity above 1 MΩ-cm to prevent short circuits if leaks occur
• pH control: Targets a slightly alkaline range (8.0-8.5) to minimize corrosion of copper and stainless steel components
• Regular sampling: Quarterly lab analysis checks for dissolved metals, biological growth, and inhibitor depletion to schedule proactive fluid replacement

A cold plate is a thermally conductive metal block—typically copper or aluminum—with internal micro-channels or fins that sits directly on top of a heat-generating component like a GPU. Coolant is pumped through the plate, absorbing heat via conduction and carrying it away to a heat exchanger. This is the most common implementation of direct-to-chip liquid cooling.

• Micro-channel designs maximize surface area for heat transfer
• Can remove 1,500W+ per socket in modern AI deployments
• Enables 30-50 kW per rack densities versus 10-15 kW for air cooling
• Often paired with thermal interface materials (TIMs) like liquid metal or high-performance pastes to eliminate microscopic air gaps

Unlike direct-to-chip methods, immersion cooling submerges entire servers or components in a thermally conductive but electrically non-conductive dielectric fluid. This approach eliminates the need for cold plates, fans, and complex plumbing at the component level.

• Single-phase immersion: Fluid remains in liquid state; heat is transferred via circulation to an external heat exchanger
• Two-phase immersion: Fluid boils at low temperatures, absorbing latent heat of vaporization; vapor condenses on a cooled coil and drips back
• Provides uniform cooling across all components, not just high-power chips
• Can achieve Power Usage Effectiveness (PUE) ratings below 1.05

A Coolant Distribution Unit is the pumping, filtration, and heat exchange system that manages the flow of coolant between the facility water loop and the technology cooling loop serving the cold plates. The CDU precisely controls flow rate, pressure, and temperature to ensure safe and efficient heat removal.

• Maintains supply coolant temperature typically between 25-45°C (77-113°F)
• Includes redundant pumps and power supplies for high availability
• Monitors conductivity and particulate levels to prevent corrosion and clogging
• Acts as the critical isolation point between facility water and sensitive IT equipment

A Rear Door Heat Exchanger is a passive or active liquid-cooled door that replaces the standard rear door of a server rack. Hot exhaust air from IT equipment passes through a fin-and-tube coil filled with chilled water, removing heat before it enters the data center ambient space.

• Passive RDHx uses facility chilled water; active RDHx includes integrated fans
• Captures 60-90% of server exhaust heat at the rack level
• Retrofittable onto existing air-cooled racks without modifying servers
• Often used as a hybrid approach alongside direct liquid cooling for components not on cold plates

Thermal Design Power represents the maximum amount of heat a component is expected to generate under a sustained, realistic worst-case workload. It is the fundamental metric that dictates cooling system design and capacity planning for AI clusters.

• Modern AI GPUs like the NVIDIA H100 have a TDP of 700W; the B200 reaches 1,000W
• TDP drives decisions on cold plate sizing, coolant flow rates, and CDU capacity
• Actual power draw can exceed TDP during transient spikes, requiring headroom in cooling design
• Future roadmaps point toward 1,500W+ per socket, making liquid cooling mandatory rather than optional

Power Usage Effectiveness is the ratio of total facility power consumption to IT equipment power consumption. It is the industry-standard metric for data center energy efficiency. A PUE of 1.0 represents a perfect facility where all power goes to compute.

• Traditional air-cooled data centers average PUE of 1.5-1.6
• Direct liquid cooling can drive PUE below 1.1 by drastically reducing fan energy and chiller load
• Warm water cooling (using coolant at 30-45°C) enables free cooling in most climates, eliminating mechanical refrigeration entirely
• Lower PUE directly translates to lower operational cost and reduced carbon footprint for AI factories