Why Coding Assistants Are Improving; OpenAI’s ‘Russian Doll’ Data Center

While everyone waits for OpenAI’s next flagship large language model, developers have been eking out more gains from today’s LLMs with some engineering elbow grease, as we covered here. That’s especially true for artificial intelligence-powered coding assistants, which are getting a lot better. 

Ofir Press, a postdoctoral researcher at Princeton’s Language and Intelligence Initiative, made some of those improvements personally while developing SWE-agent, an open-source coding assistant, and SWE-bench, a public leaderboard that evaluates the performance of various coding assistants.

Press said he realized that coding assistants—which can write new code or autocomplete existing code—have a lot of quirks that can be fixed. For instance, they often get confused when they perform an action that doesn’t produce a visible result. This can often happen with routine actions that run in the background, like software updates, that avoid producing outputs so as to not clutter a developer’s screen.

Coding assistants can also struggle to recover when they make relatively minor mistakes, such as when they insert an extra indentation or parentheses. Even small errors like those can drag a coding assistant down a rabbit hole, where it starts editing the same code snippet over and over again.