Intel's latest Habana Gaudi3 AI accelerators are poised to compete with Nvidia's H100 in both inference and training, despite older process tech and slower memory.
On paper, Intel 's Habana Gaudi3 AI accelerators don't look like they're ready to take on Nvidia 's H100 thanks to older process tech and slower HBM memory delivering fewer FLOPS. But Gelsinger's gang insists its latest parts can not only go toe-to-toe with the H100 in inference, but best it in training.
Intel's Gaudi3 accelerator boasts eight matrix math engines, 64 tensor cores, 96 MB of SRAM, and 128 GB of HBM2e memory – click to enlarge The often referenced figure is perhaps misleading. The 1,835 teraFLOPS claimed by Intel is dense floating point performance, while Nvidia is relying on sparsity to achieve its 4 petaFLOP claims. Taking this into account, Gaudi3 is only about 144 teraFLOPS slower than the H100 while offering more memory grunt.Then there's AMD's MI300X, which remains the FP8 FLOPS king, at least until Nvidia's Blackwell parts start making their way into customer's hands.
Here Gaudi3's older memory – HBM2e vs HBM3 on the H100 and MI300X, and HBM3e on the H200 – puts it in an odd spot.With eight stacks of HBM2e, Gaudi3 actually has more and faster memory than the Nvidia's H100 with its five stacks of HBM3. Despite this, the chip still falls well behind the H200 and MI300X's HBM3e memory, which deliver 4.8 GBps and 5.3 TBps of bandwidth respectively.
Each Gaudi3 accelerator features 24 200GbE interfaces, 21 for chip-to-chip comms and three for system-level networking – click to enlarge
Intel Habana Gaudi3 AI Accelerators Nvidia H100 Training Inference Process Tech Memory
United Kingdom Latest News, United Kingdom Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Bryan Habana offers brutally honest response to Ireland being 'world's best'Ireland may be regarded as the best team in the world right now, but Springbok legend Habana offers a different viewpoint
Read more »
Nvidia turns up the AI heat with 1,200W Blackwell GPUs5x the performance of the H100, but you'll need liquid cooling to tame the beast
Read more »
Asus ROG Zephyrus G14 (2024) reviewAMD Ryzen 9 8945HS | Nvidia GeForce RTX 4070 (90W) | 32GB LPDDR5X | 1TB SSD | OLED | $2,000 | £2,400
Read more »
Just how rich are businesses getting in the AI gold rush?Nvidia and Microsoft are not the only winners
Read more »
Dell adds Nvidia's next GPUs to its portfolio of AI platformsNvidia is a kingmaker, and who wouldn't want to be king?
Read more »
Nvidia: Why write code when you can string together a couple chat bots?GPU giant says NIM will eliminate dependency headaches for the low low cost of $4,500/year per GPU
Read more »