What On-Device Models Mean for Open-Source AI

When Apple on Monday showed off the artificial intelligence that will soon land in some of our iPhones, some developers praised the company for running many of the new features directly on the iPhone, rather than in the cloud as other companies do. Like Apple, which has made user privacy and security central to its marketing, these developers say they’re also concerned about privacy.

Apple’s on-device model is a small language model with 3 billion parameters, or the settings that determine how models respond to questions. (OpenAI’s GPT-4 is believed to have about 1 trillion parameters, to put that in perspective.) Apple’s model follows other small language models that can run on phones, such as Google’s Gemma and Microsoft’s Phi-2. 

These releases didn’t come out of nowhere. Independent developers have been working to run open-source AI models, such as Meta Platforms’ Llama 3 and those from Mistral, on phones, laptops and other devices for months. These developers say that they want to better control their models, partly because of their desire to ensure user privacy, but partly because they’re daunted by the high cost of using larger proprietary models, like OpenAI’s.