OpenAI’s Models Are Getting Too Smart For Their Human Teachers

In the fight to improve AI models, Anthropic and OpenAI have doubled down on two methods: letting models train on fake clones of apps—otherwise known as reinforcement learning environments or gyms—and getting experts in various fields to teach models new things, as I reported in this Tuesday story.

One problem is emerging, though: It’s getting a lot harder for human experts to stump the models and expand their knowledge in certain fields.

For example, one expert helping teach OpenAI’s o3 model last year told me that at the time, he could come up with three to four linguistics-related tasks per week that the model couldn’t do. Now that he’s working with GPT-5, which launched this summer, it’s been harder to come up with linguistic questions that the model can’t answer. He’s only been able to come up with one to two new tasks that the model can’t do per week.

“It feels like we're training models to replace ourselves,” he said.