Last week, I pointed our @poolsideai harness + new Laguna S 2.1 at a research task: one GPU, five hours, make Qwen3-1.7B better at grade-school math. No humans in the loop. Nobody peeking over its shoulder.
13% → 61% on GSM8K.
But honestly? The score was the boring part. 🧵
13% → 61% on GSM8K.
But honestly? The score was the boring part. 🧵
First, the driver: Laguna S 2.1 is a 118B total parameter MoE with just 8B activated params.
Not a frontier giant. An efficient model, running in a harness - and it did research work for hours. Keep that in mind for everything below.
Not a frontier giant. An efficient model, running in a harness - and it did research work for hours. Keep that in mind for everything below.
Have you ever babysat a long-running experiment? Checking in every 20 minutes because you don't quite trust it?
This run didn't need it. 4+ hours, 366 tool calls, the same loop over and over:
baseline → clean data → fine-tune → evaluate → adjust → repeat.
Real experiments, not one-shot prompting.
This run didn't need it. 4+ hours, 366 tool calls, the same loop over and over:
baseline → clean data → fine-tune → evaluate → adjust → repeat.
Real experiments, not one-shot prompting.
It swept learning rates and epochs, then layered iterative DPO on top — generating its own wrong answers, pairing them against correct ones, training the model to prefer correct.
It also tried self-training and an "align-to-eval" trick. Both failed. It said so and moved on. No burying the failed runs.
It also tried self-training and an "align-to-eval" trick. Both failed. It said so and moved on. No burying the failed runs.
Now, the moment that actually impressed me.
Its fast local evaluator said a checkpoint hit 67%. The official one said 43%.
Guess what most of us would do? Yes, correct — believe the 67%. I've seen seasoned engineers do this. I've been that engineer.
Its fast local evaluator said a checkpoint hit 67%. The official one said 43%.
Guess what most of us would do? Yes, correct — believe the 67%. I've seen seasoned engineers do this. I've been that engineer.
The agent did something else: it treated the disagreement as the finding.
It dug in and found the bug itself - its shortcut evaluator used a different decoding backend, diverging exactly on hard examples (clean "ANSWER: 6" vs a garbled "6/0").
It threw out its own broken metric and re-ran everything on the real one. Unprompted.
It dug in and found the bug itself - its shortcut evaluator used a different decoding backend, diverging exactly on hard examples (clean "ANSWER: 6" vs a garbled "6/0").
It threw out its own broken metric and re-ran everything on the real one. Unprompted.
Honest caveats:
- Plain fine-tuning did most of the work (13% → 58%). DPO added ~3 pts on top
- "Better at GSM8K by training on its train split" is a working pipeline, not a reasoning leap.
- Plain fine-tuning did most of the work (13% → 58%). DPO added ~3 pts on top
- "Better at GSM8K by training on its train split" is a working pipeline, not a reasoning leap.
So what is this? Not a superhuman model. Nobody needs another one of those announcements.
It's an 8B-active model, in the right harness, able to run locally, holding a multi-hour goal, debugging its own tooling, discarding its failures honestly, and shipping a verified result. Solo.
You don't need a giant to do honest research. You need a loop that refuses the nicer number 🧪
It's an 8B-active model, in the right harness, able to run locally, holding a multi-hour goal, debugging its own tooling, discarding its failures honestly, and shipping a verified result. Solo.
You don't need a giant to do honest research. You need a loop that refuses the nicer number 🧪
Can't wait to see what the community does with our model. Go try it out, it's free and open weights:
https://x.com/poolsideai/status/2079613777343848465?s=20
• • •
Missing some Tweet in this thread? You can try to
force a refresh
