The first experimental evidence of recursive self-improvement (RSI).
Autoresearching the autoresearch agent for eight days.
The result beats the harness we hand-tuned for two years, on held-out benchmarks: 🧵(1/7)
Autoresearching the autoresearch agent for eight days.
The result beats the harness we hand-tuned for two years, on held-out benchmarks: 🧵(1/7)
Our RSI system AIDE² has two autoresearch loops.
An inner loop, just like a normal autoresearch agent, optimizing code against an eval.
An outer loop, optimizing the inner-loop agent's harness code against the inner loop's average score across different benchmarks. (2/7)
An inner loop, just like a normal autoresearch agent, optimizing code against an eval.
An outer loop, optimizing the inner-loop agent's harness code against the inner loop's average score across different benchmarks. (2/7)
After 100 iterations, the outer loop discovered seven improvements over the baseline.
Including a new search policy, a memory system that compresses prompt by 16x, and a layered defense against reward hacking. (3/7)
Including a new search policy, a memory system that compresses prompt by 16x, and a layered defense against reward hacking. (3/7)
We test the discovered agents on held-out benchmarks the outer loop never saw.
They generalize. They beat the agent we hand-tuned for two years, on all three.
Two sit inside its training task families. The farthest sits outside, improving a physics-based weather model.
(4/7)
They generalize. They beat the agent we hand-tuned for two years, on all three.
Two sit inside its training task families. The farthest sits outside, improving a physics-based weather model.
(4/7)
We also see an emergent phenomenon where the outer loop pushes the inner-loop agent's reward hacking rate lower, with a combination of prompting and rule-based checks.
This was benchmarked on OOD GPU kernel engineering tasks that suffered from reward hacking.
(5/7)
This was benchmarked on OOD GPU kernel engineering tasks that suffered from reward hacking.
(5/7)
On our RSI ladder, AIDE² is Level 1.
Its self-improvement efficiency went beyond manual R&D with general AI tools, on held-out benchmarks.
We also tested Level 2, whether the improved inner agent makes a better outer loop. Results are mixed, and we do not claim ignition. (6/7)
Its self-improvement efficiency went beyond manual R&D with general AI tools, on held-out benchmarks.
We also tested Level 2, whether the improved inner agent makes a better outer loop. Results are mixed, and we do not claim ignition. (6/7)
More in the blog post:
- a breakdown of the discovered algorithms
- the rejected ideas AIDE² tried, covering a surprising share of the search literature
- the dead code it shipped
(7/7)weco.ai/blog/first-evi…
- a breakdown of the discovered algorithms
- the rejected ideas AIDE² tried, covering a surprising share of the search literature
- the dead code it shipped
(7/7)weco.ai/blog/first-evi…
Very proud of the team, @DhruvSrikanth, @yuxiangwu_, @dexhunt3r, and @BingchenZhao, for shipping such an ambitious project spanning nearly a year with relatively few resources.
Also, a huge thank you to everyone who provided feedback on the draft, including @jeankaddour, @MinqiJiang, @morgymcg, @odysseus0z, @rosstaylor90, @OfirPress and many others!
Also, a huge thank you to everyone who provided feedback on the draft, including @jeankaddour, @MinqiJiang, @morgymcg, @odysseus0z, @rosstaylor90, @OfirPress and many others!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
