Today we’re releasing research with @apolloaievals.
In controlled tests, we found behaviors consistent with scheming in frontier models—and tested a way to reduce it.
While we believe these behaviors aren’t causing serious harm today, this is a future risk we’re preparing for. openai.com/index/detectin…
In controlled tests, we found behaviors consistent with scheming in frontier models—and tested a way to reduce it.
While we believe these behaviors aren’t causing serious harm today, this is a future risk we’re preparing for. openai.com/index/detectin…
Scheming = when an AI behaves one way on the surface while hiding its true goals.
Today’s deployed systems have little opportunity to scheme in ways that could cause serious harm. The most common failures are simple deceptions—like pretending to complete a task without doing it. We’ve studied and mitigated these issues and made meaningful improvements in GPT-5 over earlier models.
But as AIs take on more complex, long-term tasks with real-world impact, the potential for harmful scheming will grow—so our safeguards and testing must grow with it.
Today’s deployed systems have little opportunity to scheme in ways that could cause serious harm. The most common failures are simple deceptions—like pretending to complete a task without doing it. We’ve studied and mitigated these issues and made meaningful improvements in GPT-5 over earlier models.
But as AIs take on more complex, long-term tasks with real-world impact, the potential for harmful scheming will grow—so our safeguards and testing must grow with it.
Typically, as models become smarter, their problems become easier to address—for example, smarter models hallucinate less and follow instructions more reliably.
However, AI scheming is different.
As we train models to get smarter and follow directions, they may either better internalize human goals or just get better at hiding their existing true goals.
The core of anti-scheming research is to distinguish between these two, which requires understanding the reasoning behind a model's behavior.
However, AI scheming is different.
As we train models to get smarter and follow directions, they may either better internalize human goals or just get better at hiding their existing true goals.
The core of anti-scheming research is to distinguish between these two, which requires understanding the reasoning behind a model's behavior.
In this new research with @apolloaievals, we found behaviors consistent with scheming in controlled tests across frontier models, including OpenAI o3 and o4-mini, Gemini-2.5-pro, and Claude Opus-4.
We can significantly reduce scheming by training models to reason explicitly, using an extension to the Model Spec that prohibits scheming. That method is called deliberative alignment.
With this technique, we can reduce covert actions by 30x for o3.
However, situational awareness complicates results.
Model spec: model-spec.openai.com/2025-04-11.html
We can significantly reduce scheming by training models to reason explicitly, using an extension to the Model Spec that prohibits scheming. That method is called deliberative alignment.
With this technique, we can reduce covert actions by 30x for o3.
However, situational awareness complicates results.
Model spec: model-spec.openai.com/2025-04-11.html
Frontier models can recognize when they are being tested, and their tendency to scheme is influenced by this situational awareness.
We demonstrated counterfactually that situational awareness in their chain-of-thought affects scheming rates: the more situationally aware a model is, the less it schemes, and vice versa.
Moreover, both RL training and anti-scheming training increase levels of situational awareness.
We demonstrated counterfactually that situational awareness in their chain-of-thought affects scheming rates: the more situationally aware a model is, the less it schemes, and vice versa.
Moreover, both RL training and anti-scheming training increase levels of situational awareness.
Our results depend on reading models’ reasoning (“chain-of-thought”), and we believe the field isn't prepared for eval-aware models with opaque reasoning.
Until better methods exist, we urge developers to preserve chain-of-thought transparency to study and mitigate scheming.
Until better methods exist, we urge developers to preserve chain-of-thought transparency to study and mitigate scheming.
This is significant progress, but we have more work to do.
We’re advancing scheming research categories in our Preparedness Framework, renewing our collaboration with Apollo, and expanding our research team and scope. And because solving scheming will go beyond any single lab, we're also exploring ways to raise awareness and collaborate including piloting cross-lab evals, launching a $500K Kaggle challenge, and urging chain-of-thought transparency industry-wide.
We’re advancing scheming research categories in our Preparedness Framework, renewing our collaboration with Apollo, and expanding our research team and scope. And because solving scheming will go beyond any single lab, we're also exploring ways to raise awareness and collaborate including piloting cross-lab evals, launching a $500K Kaggle challenge, and urging chain-of-thought transparency industry-wide.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
