Nvidia Blows Competitors Out of the Water in New Test

All eyes are on Nvidia’s quarterly earnings report today (there’s apparently even a watch party for it at a New York City bar if you want to partake in some nonsense), so it’s a good time to share a new example of the company’s technological dominance.

This morning, MLCommons, a nonprofit that evaluates artificial intelligence software and hardware, released test results that showed that Nvidia’s most advanced—and broadly available—chip, the H200, blew rivals out of the water when it came to powering large language models. We’re talking about what’s known in AI-land as inference, which includes generating or summarizing text, rather than the computer power required to train the models. 

For instance, Nvidia’s H200s handled a large language model developed by Meta Platforms 44% faster than Advanced Micro Devices’ most advanced chip, the Instinct MI300X, according to the test. An AMD spokesperson pointed out that the MI300X was neck-and-neck with Nvidia’s H100 in the same test and noted that this is AMD’s first time submitting its MI300X chip. We’re not sure why that makes a difference, although AMD may learn how to better optimize its chips for these kinds of tests over time.