High-performance data center and AI workloads are power-intensive, outpacing efficiency improvements in air-cooling technology. Power requirements for AI model training are increasing to over 100kW per rack in some use cases, of which air-cooling fans alone account for up to 15% of server power consumption. To improve cooling capacity and reduce long-term capital expenditure, the market is turning to alternative methods to air-cooling. Liquid and immersion cooling can be implemented for all components in the AI and HPC architecture, including XPUs, storage, and networking.

Data Center Liquid Cooling

Dell’Oro Group reports “mainstream adoption of liquid cooling starting in the second half of 2024… to materialize over the next five years (2024-2028) in a market opportunity totaling more than $15 B.” To support this nascent and fast-growing market, Amphenol has developed new product lines compatible with alternative technology such as liquid and immersion cooling.

1.  Liquid cooling:

Liquid cooling utilizes the greater thermal conductivity of water or other fluids and can be up to 3000 times more effective than air. The coolant fluid is cycled via active or passive heat exchangers, which are commonly installed at the rear of the rack. With a proven track record in workstations and desktop gaming applications, liquid cooling is being implemented by data center architects at the rack level, to remove heat from all or select equipment shelves.

Liquid cooling can also be selectively implemented at the chip or component level, directly targeting primary heat-generating components to remove excess heat through single-phase cold plates or two-phase evaporation units. These cooling technologies can remove about 70-75% of the heat generated by the target components, with the remainder to be removed by conventional air-cooling appliances.

HPC and AI applications are the primary factor driving the adoption of liquid cooling. Meanwhile, pluggable copper and optical IO module power consumption exceed MSA-specified limits, necessitating more effective cooling methods for front-panel pluggable form-factor cages. Conventional air-cooled heat sinks are replaced with liquid-cooled cold plates, which directly contact the top surface of each pluggable module to efficiently maintain a reasonable operating case temperature.

Amphenol’s liquid-cooled solution for front-panel pluggables is available for OSFP in a 1x8 ganged configuration and QSFP-DD in a 2x8 ganged and stacked configuration, enabling increased switch density. These cages are complaint with respective MSA specifications and are compatible with OSFP and QSFP-DD products on the general market.

2.  Immersion cooling:

For immersion cooling, components, servers or whole racks are submerged in a thermally conducive dielectric fluid, formulated to be a more efficient heat conductor than air or water. Immersion cooling at the rack level eliminates conventional air-cooling fans and systems, as well as the growing need for separate air and liquid cooling systems, simplifying data center infrastructure and maintenance.

Immersion cooling can significantly simplify cooling infrastructure and reduce the number of power-hungry cooling appliances, thereby improving energy efficiency and lowering Power Usage Effectiveness (PUE). The typical data center PUE across the globe ranges between 1.6 and 1.9, while immersion cooling can achieve PUE as low as 1.01. Researchers are investigating immersion methods that can reduce power consumption in power- and compute-intensive AI clusters and HPC networks by as much as 70%.

In single-phase immersion cooling, the selected coolant fluid has a high boiling point and remains in the liquid state throughout the operation. Cool liquid surrounds the equipment and is siphoned off as it rises to the top. The heated fluid is cooled using a heat exchanger and returned to the immersion cooling intake.

Two-phase immersion cooling utilizes coolant fluid with a low boiling point and capitalizes on the phase change from liquid to vapor to capture excess heat. The vapor is captured and cooled with a condensing coil and returned to the system in liquid form.

Immersion cooling presents many challenges for the interconnect designer. Immersion coolant fluids must be individually evaluated for their potential impact on connector and PCB materials. Long-term submersion in hydrocarbon or silicone-based fluid may lead to corrosion or degradation of a variety of materials such as PCB composites, metallic plating, solder joints, cable insulation, or adhesives.

Conventional connector design leverages the dielectric properties of air for impedance control. Substituting air with any number of immersion coolant fluids available on the market greatly affects the effective dielectric properties throughout the connector and thereby its signal integrity characteristics. In simulation, an immersion fluid with dielectric constant Dk = 1.0 (shown below in red) best emulates the target performance in air (Dk = 1.0) (in blue), while immersion fluids with Dk of 1.8 (green), 2.6 (purple), and 3.4 (yellow) show increasingly worse performance in differential impedance, return loss, and insertion loss. A high-speed interconnect designed for ideal performance in air must therefore be reengineered for submersion in dielectric fluid in order to maintain superior signal integrity at the higher frequencies.

 

Difference Impedance

Amphenol has developed the ExaMAX2® ARK Connector for ideal performance characteristics in immersion-cooled systems.  It is specifically engineered to demonstrate performance on par with that of the EXAMAX2® Connector Series performance in air. The EXAMAX2® ARK Product Line is a derivative product of the broadly adopted ExaMAX® Product Series and is mate and footprint compatible with EXAMAX2® Connectors.

Related Videos: