What If LLMs Could Continue Learning?

Writer cofounders May Habib and Waseem Alshikh. Courtesy of Writer.

Here’s a question of paramount importance in AI: what if large language models could continue learning new information even after they’re done training?

That’s the question that researchers at Writer, a $2 billion-valuation startup developing AI tools for enterprises, found themselves asking six months ago. Those researchers have now developed a new type of LLM, called “self-evolving LLMs,” that they claim can continue learning and updating its parameters, or the connections between parts of the model that determine how it answers questions, even after it’s deployed.

The development, if it proves as good as it sounds, might provide a solution to the growing concerns that traditional methods to improve LLMs aren’t working as well as they used to. Those questions have prompted researchers at labs like OpenAI, Google and Meta Platforms to experiment with new methods to keep the “scaling laws” party going.