How Real is the GPU Shortage?

Startups have gone to extreme measures this year to get a hold of graphics processing units, the rare chips needed to train and run AI models. They’ve tried everything from raising capital from cloud providers like Microsoft, Google and Amazon to building bots that scour hyperscalers’ websites in search of open servers.

One AI founder, though, doesn’t think the GPU shortage is as bad as everyone has made it out to be. The real problem isn’t that founders don’t have enough chips, but rather, that they don’t know how to use the ones they do have efficiently, said Gennady Pekhimenko, co-founder and CEO of CentML. 

Pekhimenko has a good reason to make this argument: His startup helps companies optimize their AI models for different types of chips. But he’s not the only one making this case. Other founders have recommended, for instance, running smaller open-source models on cheaper, older-generation chips like Nvidia V100s. A growing number of startups are finding creative workarounds like this to make do with the chips they have.

One is Dragonfruit, which builds security camera technology to monitor retail store operations and prevent shoplifting. Because the company had to integrate its software into hundreds of cameras in locations that often had bad Internet connections, it was impossible to use expensive, top-of-the-line chips from Nvidia via the cloud. Instead, Dragonfruit turned to Apple’s M1 chip and Mac Mini computer, which run near each of the camera locations, said founder and CEO Amit Kumar.