Amazon has found its 'own way' to cool down Nvidia's AI graphics cards

Amazon 's cloud division has reportedly developed its own hardware to cool next-generation Nvidia graphics cards for artificial intelligence (AI) workloads. This internal solution addresses the significant energy consumption and heat generation associated with Nvidia's GPUs, which are vital for AI workloads. Dave Brown, Vice President of Compute and Machine Learning Services at Amazon Web Services (AWS), stated in a YouTube video that commercially available cooling equipment was not suitable and building data centres with widespread liquid cooling would have taken too much time. This led Amazon to develop its methods that can better manage the heat from these power-intensive Nvidia GPUs.

What the AWS VP said about these tools developed for Nvidia GPUs

Talking about the cooling equipment available for AI GPUs in the video, Brown said: “They would take up too much data centre floor space or increase water usage substantially. And while some of these solutions could work for lower volumes at other providers, they simply wouldn’t be enough liquid-cooling capacity to support our scale.”

So, instead of relying on conventional solutions, Amazon engineers developed the In-Row Heat Exchanger (IRHX), a cooling system that can be integrated into both existing and future data centres. Previously, these traditional air cooling methods have been sufficient for earlier Nvidia chip generations.



In a blog post, Brown also confirmed that AWS customers can now access this updated infrastructure through new P6e computing instances. These new offerings support Nvidia’s high-density computing architecture, particularly the GB200 NVL72, which consolidates 72 Nvidia Blackwell GPUs into a single rack for training and deploying large AI models.

Previously, similar Nvidia GB200 NVL72-based clusters were available via Microsoft and CoreWeave. AWS, as the leading global cloud infrastructure provider, continues to enhance its capabilities.

Amazon has a history of developing its infrastructure hardware, including custom chips for general computing and AI, along with in-house-designed storage servers and networking equipment.

This approach reduces reliance on external vendors and can improve profitability. AWS posted its highest operating margin since at least 2014 during the first quarter, contributing significantly to Amazon’s overall net income.

Microsoft, the second-largest cloud provider, has also moved into custom hardware. In 2023, it introduced a cooling system called Sidekicks, tailored for its Maia AI chips.