Where LLMs Are Falling Short

I’m back from the International Conference on Machine Learning in Vancouver, one of the biggest annual meetups for artificial intelligence researchers. And this year, the conference underscored all the ways in which large language models are still falling short of everyone’s expectations, despite the immense progress that got us to this point.

Researchers even called into question some of the most promising techniques that have gained popularity over the last year, such as chain-of-thought reasoning, or asking the models to describe how they arrived at an answer—their “thoughts,” so to speak.

For instance, one research paper presented at the event explained how chain-of-thought can actually hurt model performance on certain tasks due to “overthinking.” In one example, models were shown strings of letters that followed some rule that the model didn’t know. Then the models were shown another string of letters and asked whether it followed the unknown rule or not. 

Humans typically perform better on this task when they’re told to go with their gut feeling. However, the models performed worse when asked to explain their reasoning. Models are great at finding patterns, but when there are so many possibilities for what the unknown rule or pattern might be, they tend to overthink and end up at the wrong answer, the paper argued.