1. Cooling Technology Comparison and Principles
| Liquid Cooling vs Air Cooling | ||
| Air Cooling | Liquid Cooling | |
| Heat exchange medium | Air | Liquid |
| Drive components | Fan | Remove fans |
| Heat dissipation capacity | General heat dissipation capacity | Good heat dissipation |
| Energy saving and consumption reduction | PUE value less than 2 | PUE value within 1.2 |
| Noise | High noise | Remove fans, low noise |
| Construction cost | Cabinets can only be arranged in low density, and the cabinets occupy a large area of the computer room, requiring traditional precision air conditioning and hot and cold channel design | Can bring high-density cabinet layout design, reduce the occupied computer room area, low PUE means smaller scale of power supply, distribution and backup infrastructure |
| Site selection | High requirements for environmental climate and power factors | Not restricted by air quality, climate and energy policies, can be used all over the world |
1.1 Air Cooling Technology
Diagram of Air Cooling System Principles
Air cooling technology is the mainstream cooling technology in data centers. The heat dissipation principle of air-cooled radiators is to conduct the heat generated by the heat-generating object to the heat sink with larger heat capacity and heat dissipation area through the metal heat sink that is in close contact with the heat-generating object (for computers, it is the CPU, GPU and other semiconductor chips), and then use the fan’s diversion effect to make the air pass through the heat sink surface quickly, accelerating the heat convection between the heat sink and the air, that is, forced convection heat dissipation.
1.2 Liquid Cooling Technology
| Comparison of Three Liquid Cooling Technologies | |||
| Cold plate type | Immersion type | Spray type | |
| Cost | The cold plate requires many specifications, most of which need to be customized separately, and the cost is relatively high | Uses more coolant, with medium cost | By modifying old servers and cabinets to add necessary devices, the cost is relatively low |
| Maintainability | Excellent | Poor | Medium |
| Space utilization | High | Medium | Highest |
| Compatibility | No direct contact with the motherboard and chip module, the material compatibility is strong | Direct contact, poor material compatibility | Direct contact, poor material compatibility |
| Ease of installation | Does not change the original form of the server and retains the existing server motherboard, which is easy to install | Changes the original structure of the server motherboard, needs to be reinstalled | Does not change the original form of the server motherboard, easy to install |
| Recyclability | Uses dual-circuit loop circulation to achieve secondary use of the refrigerant and reduce operating costs | Circulates through outdoor cooling devices to reduce operating costs | Use a circulation pump to achieve resource reuse and reduce operating costs |
Liquid cooling technologies include cold plate, immersion, and spray types. Among them, cold plate liquid cooling technology has strong application advantages in maintainability, space utilization, and compatibility; but in terms of cost, due to its individually customized cold plate device, the cost of technology application is relatively high. Spray liquid cooling technology greatly reduces the construction cost of data center infrastructure by transforming old servers and cabinets. Compared with the other two technologies, although the immersion technology has poorer maintainability and compatibility, it has better performance in space utilization and recyclability, reducing the energy consumption of data centers.
Principle Diagram of Cold Plate Liquid Cooling System
Cold plate liquid cooling is a non-contact liquid cooling technology. This technology indirectly transfers the heat of the heating device to the cooling liquid enclosed in the circulation pipeline through the liquid cooling plate (usually a closed cavity made of heat-conducting metals such as copper and aluminum), and takes away the heat through the cooling liquid. The cold plate liquid cooling system is mainly composed of a cooling tower, CDU, primary and secondary side liquid cooling pipelines, cooling medium, and liquid cooling cabinet; the liquid cooling cabinet contains a liquid cooling plate, liquid cooling pipelines in the equipment, fluid connectors, and liquid distributors.
Cold plate liquid cooling heat dissipation principle:
1.The liquid cooling plate is bonded to the chip;
2.The heat of the chip equipment is transferred to the liquid cooling plate through heat conduction, and the working fluid enters the cold plate under the drive of the CDU circulation pump, and then absorbs heat in the liquid cooling plate through enhanced convection heat exchange.
Principle Diagram of Single-Phase Immersion Liquid Cooling System
Liquid immersion cooling is a contact liquid cooling technology. This technology uses coolant as the heat transfer medium, completely immerses the heat generating device in the coolant, and the heat generating device is in direct contact with the coolant and performs heat exchange. The outdoor side of the immersion liquid cooling system includes a cooling tower, a primary side pipe network, and a primary side coolant; the indoor side includes a CDU, an immersion cavity, IT equipment, a secondary side pipe network, and a secondary side coolant. During use, the IT equipment is completely immersed in the secondary side coolant, so the secondary side circulating coolant needs to use a non-conductive liquid, such as mineral oil, silicone oil, fluorinated liquid, etc. According to whether the coolant changes in phase during the heat exchange process, it can be divided into single-phase immersion liquid cooling and two-phase immersion liquid cooling.
Among them, the secondary side coolant of the single-phase immersion liquid cooling technology as the heat transfer medium only undergoes temperature changes during the heat transfer process, and there is no phase change. The process completely relies on the sensible heat change of the material to transfer heat.
Two-Phase Immersion Liquid Cooling System
In two-phase immersion liquid cooling, the secondary coolant, which serves as a heat transfer medium, undergoes a phase change during the heat transfer process, and transfers heat by relying on the latent heat change of the substance. Its heat transfer path is basically the same as that of single-phase immersion liquid cooling. The main difference is that the secondary coolant only circulates inside the immersion cavity, and the top of the immersion cavity is a gaseous area and the bottom is a liquid area: the IT equipment is completely immersed in the low-boiling-point liquid coolant, and the liquid coolant absorbs the heat of the equipment and boils. The high-temperature gaseous coolant produced by vaporization will gradually gather at the top of the immersion cavity due to its low density, and condense into a low-temperature liquid coolant after heat exchange with the condenser installed on the top, and then flow back to the bottom of the cavity under the action of gravity to achieve heat dissipation of the IT equipment.
Principle Diagram of Spray Liquid Cooling System
Spray liquid cooling is a form of liquid cooling that sprays precisely on chip-level devices and directly sprays coolant onto heat-generating devices or heat-conducting elements connected to them by gravity or system pressure. It is a direct contact liquid cooling system. The spray liquid cooling system is mainly composed of a cooling tower, CDU, primary and secondary liquid cooling pipelines, cooling medium and spray liquid cooling cabinet; the spray liquid cooling cabinet usually includes a pipeline system, a liquid distribution system, a spray module, a liquid return system, etc.
Principle of the spray liquid cooling system: the coolant cooled in the cold distribution unit is pumped to the inside of the spray cabinet through the pipeline; after entering the cabinet, the coolant directly enters the liquid distribution device corresponding to the server through the liquid distributor, or the coolant is transported to the liquid inlet tank to provide a fixed amount of gravitational potential energy to drive the coolant to spray through the liquid distribution device; the coolant is sprayed and cooled through the heat-generating device in the IT equipment or the heat-conducting material connected to it; the heated coolant will be collected through the return tank and pumped to the cold distribution unit for the next refrigeration cycle.
1.3 Evolution of Cooling Methods in Rack Power
Evolution of Cooling Methods Diagram
The power of a single cabinet exceeds the threshold of air cooling, and liquid cooling is the general trend. According to the Vertiv technology white paper, air cooling is generally suitable for power densities below 20kW/cabinet, and liquid cooling has obvious advantages above 20kW. Without considering factors such as power modules and network modules, assuming that 6 AI training servers can be placed in a single rack, the estimated power of a single rack can reach 37.8kW (2 CPUs with a power consumption of 700w and 8 GPUs with a power consumption of 5600w); considering the heat dissipation of other modules in the cabinet, the actual power of a single cabinet will be higher. For AI training and inference server cabinets, the power of a single cabinet has exceeded the power density range that can be covered by air cooling, and liquid cooling has become a general trend.
2. Evolution of Cooling Methods Diagram
2.1 Chip Power Surge: The Shift to the Era of Liquid Cooling
| Comparison of Three Cooling Systems | |||
| Traditional air cooling | Cold plate liquid cooling | Immersion liquid cooling | |
| Cooling performance | AA | AAA | AAAA |
| PUE | 1.5-1.9 | 1.2-1.3 | 1.1 |
| Noise | High | Low | Very low |
| Technology maturity | AAAAA | AAA | AA |
| Maintenance cost | AAAAA | AAAA | AA |
| Rack density | <10kw, cost increases when it exceeds 15kw | 15kw-100kw | 30kw-100kw |
Compared with air cooling, liquid cooling has more advantages in heat dissipation, energy consumption, noise, and maintenance costs. The heat capacity of water is 4000 times that of air, and the thermal conductivity is 25 times that of air. At the same flow rate, it can more effectively reduce the chip temperature. At the same time, liquid cooling technology can evenly cover the chip surface and improve system stability. The liquid cooling system only needs a low fan speed to maintain heat dissipation, so the noise can be reduced by more than 60% compared with air cooling during operation. In addition, the lack of easily worn parts makes the liquid cooling system have a longer service life.
Liquid cooling will replace air cooling as the best choice. The power of a single cabinet for air cooling is 0-30kw, and the power of a single cabinet for liquid cooling is 30-200kw, of which the cold plate is 30-80kw and the immersion is 80-200kw. At the 2024 GTC conference, NVIDIA released the NVL72 cabinet with a power of 120kw. With the continuous upward trend of the power of a single cabinet, air cooling has gradually failed to meet the heat dissipation requirements, and liquid cooling will become the mainstream trend.
2.2 Driving Force Three: Rise of AI Servers Makes Liquid Cooling Solutions More Cost-Effective
AI computing power demand drives total server market shipments. Since 2019, my country’s server shipments have maintained a steady upward trend, and are expected to reach 4.55 million units in 2024, a year-on-year increase of 1.3%. Although global shipments will decline in 2023 due to the penetration of high-cost AI servers and the postponement of general server updates, the rapid growth of the AI server market is expected to drive total market shipments, which are expected to reach 13.654 million units in 2024, a year-on-year increase of 19.8%.
The proportion of AI server shipments has increased year by year, and the market prospects for liquid-cooled servers are promising. In 2023, the global AI server shipments accounted for 10.4%, and the domestic share was 7.9%, maintaining a steady growth since 2020. It is expected that the global AI server share will increase to 15% in 2026. According to IDC, my country’s liquid-cooled server shipments in 2023 will be 161,000 units, accounting for 45% of the AI server market shipments. With the popularization and deployment of liquid-cooled data centers and the support of national policies, server shipments are expected to continue to grow.
3. Landscape of the Liquid Cooling Market and Industry Chain
3.1 Rapid Growth in the Energy Storage Industry Drives Increased Demand for Thermal Control Systems
The rapid development of the global large-scale storage market is the main source of demand for temperature control, especially liquid cooling temperature control. In terms of large-scale storage in the domestic market, based on the expectation of annual new energy power generation installed capacity, as well as the growth of the average energy storage configuration rate and configuration time of new projects, we estimate that from 2023 to 2025, the new installed capacity of domestic pre-meter energy storage will reach 31, 52, and 83 GW respectively.
In terms of global market storage, taking into account the energy storage construction status and construction needs of major markets such as China, the United States, the European Union, Australia, and Japan, we estimate that from 2023 to 2025, the global new installed capacity of pre-meter energy storage will reach 90GW, 143GW, and 212GW, respectively.
In terms of global market storage, taking into account the energy storage construction status and construction needs of major markets such as China, the United States, the European Union, Australia, and Japan, we estimate that from 2023 to 2025, the global new installed capacity of pre-meter energy storage will reach 90GW, 143GW, and 212GW, respectively.
