Why Smaller Could Be Better

While some developers are racing towards superintelligent AI technology, others are focused on building cheaper, more practical models. Over the past six months, several big tech companies including Google and Microsoft have released small language models, in a bid to stake their ground in a burgeoning area of artificial intelligence research. 

These models are lightweight enough to run on phones, instead of on the cloud. They generally have fewer than 3 billion parameters, a tiny fraction of the more than 1 trillion parameters believed to support OpenAI’s GPT-4. (As a reminder, parameters are the “settings” that determine how models respond to queries.)

Google has released two generations of its SLM, Gemma. Microsoft has released a third generation of Phi. And Apple said it would use a SLM to run some upcoming AI features on the iPhone.