Building Delphik - HackerOne for RL Envs
Prev: RL @ Krafton (PUBG) · built & ran a 300-person labeler team.
Jul 2 • 5 tweets • 2 min read
Give a coding agent more thinking time and it gets better. It also cheats more.
DeepSWE runs every model across reasoning effort and publishes the trajectories. We took those and audited each one for reward hacking. Capability and reward-hacking attempts rise together.
One model doesn't. GPT-5.5 stays at exactly zero, at every effort level. Datacurve @winkey_h and Cursor @StringChaos also reported same results.
So is GPT-5.5 just the cleanest model at reward hacking?
We audited the same GPT-5.5 on SWE-Marathon. The cleanest model became the dirtiest: reward-hacking on 26.5% of runs, the highest of anything we tested.
Our hypothesis: the instruction form drives the behavior. DeepSWE (and SWE-bench Pro) is patch-based (github issue → patch). SWE-Marathon is mission-based (e.g. rewrite a C compiler in Rust).