15,000 GPUs per DC, in hosts packing eight apiece, plus nine NICs – helped by switches with custom heat sinks
Alibaba Cloud has revealed the design of an Ethernet-based network it created specifically to carry traffic for training large language models – and has used in production for eight months.
Alibaba Cloud's homebrew alternative, named"High Performance Network" ,"avoids hash polarization by decreasing the occurrences of ECMP, but also greatly reduces the search space for path selection, thus allowing us to precisely select network paths capable of holding elephant flows." But while Alibaba Cloud's NPM design is based around Ethernet, it still uses Nvidia tech. The GPU champ's NVlink is used for the intra-host network and its"rail-optimized" design approach that sees each network interface card connect to a different set of top-of-rack switches is also in place.Alibaba Cloud details storage tech that's doubled its VMs per host
If the chips warm beyond 105°C, the switches can shut down. Alibaba Cloud could not find a switch vendor that offered cooling capable of keeping chips below 105°C."By optimizing the wick structure and deploying more wicked pillars at the center of the chip, heat could be carried out more efficiently," the paper explains.All of the above is built into"pods" that house 15,000 GPUs apiece, each of which resides in a single datacenter building.
United Kingdom Latest News, United Kingdom Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Alibaba Cloud unleashes thousands of Chinese AI models to the worldLike Bedrock or Azure OpenAI Studio – but with the added fun of geopolitical risk
Read more »
Alibaba Cloud built its edge network harwdare on Intel Ethernet ASICsCustom 2U boxen with Tofino inside are in production and mean Chinese cloud has more space for servers
Read more »
ASUS quietly built supercomputers, datacenters and an LLM. Now it's quietly selling them all togetherThe plan is a slow build – not a breakout into enterprise tech
Read more »
In support of Internet projectsCompany offering cloud-based hosting, mail, SaaS, backup, CDN and other services celebrates 27th birthday
Read more »
Earthcare cloud mission launches to resolve climate unknownsEurope's Earthcare satellite will tell us if the planet could lose the cooling effect of clouds.
Read more »
Satellite that aims to study how clouds affect Earth’s climate set for launchEarthCARE will study how aerosols such as dust and smoke play a role in cloud formation.
Read more »