Microsoft is beginning to immerse its servers in liquid to improve their performance and energy efficiency. Now a server rack is used for production loads in what looks like a liquid bath. This immersion process has been around in the industry for a few years, but Microsoft claims it is “the first cloud provider to run two-stage immersion cooling in a production environment.”
Cooling works by completely immersing the server racks in a specially designed non-conductive fluid. Fluorocarbon-based liquid works by removing heat when it hits components directly and the liquid reaches a lower boiling point (122 degrees Fahrenheit or 50 degrees Celsius) to condense and fall back into the bath as a raining liquid. This creates a closed-loop cooling system, which reduces costs, as no energy is needed to move the liquid through the tank and no cooler is needed for the condenser.
“It’s essentially a bathtub,” explains Christian Belady, vice president of Microsoft’s Advanced Data Center Development group, in an interview with The edge. “The rack will be placed inside the bathtub, and what you will see will be simmering as if it were boiling in your pot. The boil in the pot is 100 degrees Celsius, and in this case, 50 degrees Celsius. “
Crypto miners have used this type of liquid cooling in recent years to mine bitcoins and other cryptocurrencies. This method inspired Microsoft to test its use in recent years, using it to test against cloud demand spikes and intensive workloads for applications such as machine learning.
Most data centers are air-cooled at the moment, using outside air and cooling it by dropping it to temperatures below 35 degrees Celsius through evaporation. This is known as swamp cooling, but it uses a lot of water in the process. This new liquid bath technique is designed to reduce the use of water. “It will potentially eliminate the need to consume water in data centers, so that’s really important to us,” says Belady. “It’s really about driving less and less impact wherever we land.”
This server tub also allows Microsoft to package hardware more tightly, which should reduce the amount of space required in the long run compared to traditional air cooling. Microsoft is initially testing this with a small in-house production workload, with plans to use it more widely in the future. “You are in a small data center and we are looking for the value of a rack,” says Belady. “We have a phased approach and our next phase will be very soon with multiple racks.”
Microsoft will primarily study the reliability implications of this new cooling and with what types of bursty workloads it could even help for cloud and artificial intelligence demand. “We expect much better reliability. Our work with the Project Natick program a few years ago really demonstrated the importance of removing moisture and oxygen from an environment, ”explains Belady.
The Natick project saw Microsoft sink an entire data center to the bottom of the Scottish Sea, sinking 864 servers and 27.6 petabytes of storage into the water. The experiment was a success, and Microsoft had only one-eighth the failure rate of a terrestrial data center. “What we expect with immersion is a similar trend, because the fluid displaces oxygen and moisture, and both create corrosion … and those are the things that create failure in our systems,” says Belady.
Some of this work is also related to Microsoft’s environmental commitment to address water scarcity. The company has committed to replenishing even more water than it uses for its global operations by 2030. This includes Microsoft using an on-site rainwater harvesting system in its offices and collecting condensation from air conditioners to water plants. However, Microsoft withdrew almost 8 million cubic meters of water from municipal systems and other local sources in 2019, compared to just over 7 million in 2018.
Microsoft’s effort to address its water use will be extremely challenging given its trend toward higher water use, but projects like two-phase immersion will certainly help if implemented more widely. “Our goal is to reach zero water use,” says Belady. “That’s our metric, that’s why we are working.”