The Evolution of Language Models
Large language models (LLMs) like ChatGPT can write an essay or plan a menu almost instantly. But until recently, it was also easy to stump them. The models, which rely on language patterns to respond to users’ queries, often failed at math problems and were not good at complex reasoning. Suddenly, however, they’ve gotten a lot better at these things.
A new generation of LLMs known as reasoning models are being trained to solve complex problems. Like humans, they need some time to think through problems like these — and remarkably, scientists at MIT’s McGovern Institute for Brain Research have found that the kinds of problems that require the most processing from reasoning models are the very same problems that people need take their time with.
Human-Like Approach to Problem Solving
The researchers, led by Evelina Fedorenko, an associate professor of brain and cognitive sciences and an investigator at the McGovern Institute, conclude that in at least one important way, reasoning models have a human-like approach to thinking. That, they note, is not by design.
“People who build these models don’t care if they do it like humans. They just want a system that will robustly perform under all sorts of conditions and produce correct responses,” Fedorenko says. “The fact that there’s some convergence is really quite striking.”
How Reasoning Models Work
Like many forms of artificial intelligence, the new reasoning models are artificial neural networks: computational tools that learn how to process information when they are given data and a problem to solve. Andrea Gregor de Varda, a K. Lisa Yang ICoN Center Fellow and a postdoc in Fedorenko’s lab, explains that reasoning models work out problems step by step.
“At some point, people realized that models needed to have more space to perform the actual computations that are needed to solve complex problems,” he says. “The performance started becoming way, way stronger if you let the models break down the problems into parts.”
To encourage models to work through complex problems in steps that lead to correct solutions, engineers can use reinforcement learning. During their training:
- Models are rewarded for correct answers
- They are penalized for wrong ones
- Models explore the problem space themselves
- Actions leading to positive rewards are reinforced
Measuring the Cost of Thinking
The research involved giving reasoning models and human volunteers the same set of problems, tracking not just whether they got answers right, but also how much time or effort it took them. For humans, this meant measuring response time down to the millisecond. For models, researchers tracked tokens – part of the model’s internal chain of thought.
Both humans and reasoning models were asked to solve seven different types of problems, including numeric arithmetic and intuitive reasoning. The results showed that:
- The harder a problem was, the longer it took people to solve it
- The longer it took people to solve a problem, the more tokens a reasoning model generated
- Problem classes that humans took longest to solve were the same classes requiring the most tokens for models
This striking match in the costs of thinking demonstrates one way in which reasoning models are thinking like humans, though the researchers note this doesn’t mean the models are recreating human intelligence completely.
Visit MIT News for more information about this groundbreaking research
Leave a Reply