Some reflection on what today's reasoning launch really means:
New Paradigm
I really hope people understand that this is a new paradigm: don't expect the same pace, schedule, or dynamics of pre-training era.
I believe the rate of improvement on evals with our reasoning models has been the fastest in OpenAI history.
It's going to be a wild year.
Generalization across Domain
o1 isn't just a strong math, coding, problem solving, etc. model but also the best model I've ever used for answering nuanced questions, teaching me new things, giving medical advice, or solving esoteric problems.
This shouldn't be taken for granted!
Safety by Reasoning
The fact that our reasoning models also improve on safety behavior and safety reasoning is very much non-trivial.
For years (a decade?) the boogeyman of the AI world was reinforcement learning agents which were incredibly adept at game playing but completely incapable of reasoning or understanding human values!
This is a strong point of evidence against this.
Scaling inference-time compute can compete with scaling training compute!
The fact that o1-mini is better than o1 on some evals is very very remarkable. The implications of this I'll leave as an exercise for the reader.
Multimodal Reasoning
It's kind of crazy that reasoning improves on multimodal evals as well! See MMMU and MathVista: these aren't small improvements.
New Paradigm
I really hope people understand that this is a new paradigm: don't expect the same pace, schedule, or dynamics of pre-training era.
I believe the rate of improvement on evals with our reasoning models has been the fastest in OpenAI history.
It's going to be a wild year.
Generalization across Domain
o1 isn't just a strong math, coding, problem solving, etc. model but also the best model I've ever used for answering nuanced questions, teaching me new things, giving medical advice, or solving esoteric problems.
This shouldn't be taken for granted!
Safety by Reasoning
The fact that our reasoning models also improve on safety behavior and safety reasoning is very much non-trivial.
For years (a decade?) the boogeyman of the AI world was reinforcement learning agents which were incredibly adept at game playing but completely incapable of reasoning or understanding human values!
This is a strong point of evidence against this.
Scaling inference-time compute can compete with scaling training compute!
The fact that o1-mini is better than o1 on some evals is very very remarkable. The implications of this I'll leave as an exercise for the reader.
Multimodal Reasoning
It's kind of crazy that reasoning improves on multimodal evals as well! See MMMU and MathVista: these aren't small improvements.
To be clear I'm not one of the contributors to the o1 project: this has been the absolutely incredible work of the reasoning & related teams.
The rate of progress has just been faster than anything I've ever seen: it's absurd how fast the team has climbed the scaling OOMs just after discovering this paradigm.
The rate of progress has just been faster than anything I've ever seen: it's absurd how fast the team has climbed the scaling OOMs just after discovering this paradigm.
Less seriously now:
I do want to also give a word of caution to the schizos, the hypemen, the fans and the haters:
This is a new paradigm. As with all nascent projects will be holes, bugs, issues to fix. Don't expect everything to be perfect instantly!
But you should take the rate of progress, the fact that we're solving problems that seemed miles away in the pretraining scaling laws, the fact that we now have visibility into solving many of the things which people have said LLMs could never do.
There's lots of quirks and benefits of the pretraining paradigm that might not exist in the reasoning paradigm, and vice versa. As a random example, I do believe there will be more examples of inverse scaling here than in the pre-training world (in which there were surprisingly few).
Onwards!
I do want to also give a word of caution to the schizos, the hypemen, the fans and the haters:
This is a new paradigm. As with all nascent projects will be holes, bugs, issues to fix. Don't expect everything to be perfect instantly!
But you should take the rate of progress, the fact that we're solving problems that seemed miles away in the pretraining scaling laws, the fact that we now have visibility into solving many of the things which people have said LLMs could never do.
There's lots of quirks and benefits of the pretraining paradigm that might not exist in the reasoning paradigm, and vice versa. As a random example, I do believe there will be more examples of inverse scaling here than in the pre-training world (in which there were surprisingly few).
Onwards!
This is something to remember this is not gpt-o1, it is o1, a new thing.
https://x.com/rapha_gl/status/1834288686986301502
• • •
Missing some Tweet in this thread? You can try to
force a refresh
