Thread by @tikgiau

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision.

EdgeBench is built with three ingredients:

- 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games.
- ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours.
- 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement.

After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges:

- 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function.
- 🧠 This phenomenon can be explained by an elegant theory of graph exploration.

We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings!

Blog edge-bench.org
Paper edge-bench.org/paper.pdf
GitHub github.com/ByteDance-Seed…
Dataset huggingface.co/datasets/ByteD…

Details below 👇🧵

[2/n] EdgeBench covers real work across six capability families. It includes 134 day-long tasks spanning scientific and ML problems, systems and software engineering, optimization, professional knowledge work, formal math, and interactive games. Each task gives agents at least 12 hours in an executable environment with informative real-world feedback, while recorded human expert effort averages 57.2 hours per task.

[3/n] Agents continuously learn from environments and improve their performance in EdgeBench. The representative curves below, drawn from all six capability families, show that agents continually turn environmental feedback into better artifacts, strategies, and final outcomes.

[4/n] We average learning curves of different models across 134 tasks. The noisy task-specific trajectories collapse into a simple log-sigmoid, with high precision and mean R^2=0.998.

[5/n] Here's a fun theory of learning from environments: a task's score consists of many tiny units, sitting on a graph. Learning advances like a frontier: each node unlocked reveals new unseen neighbors, so progress compounds; but the shrinking pool of unexplored nodes sets the limit. The pace of the frontier depends on both.

In fact, this heuristic admits a formal proof. Let the fraction of explored nodes be x, so the unexplored fraction is (1 − x). Our theory says the frontier expands at a rate proportional to the product x(1 − x). Taking the natural time scale to be u = log(t), the differential equation dx/du = c·x(1 − x) solves to exactly the log-sigmoid law.

More importantly, we prove that even if each individual task's progress is jagged, the log-sigmoid law still emerges in the benchmark average over many tasks. This gives an elegant explanation of the phenomenon we observed.

[6/n] We evaluated model releases from September 2025 to May 2026, using performance improvement within 2 hours as the learning-speed metric. The frontier trend shows that AI learning speed from environments roughly doubles every three months.

[7/n] Below we show a 12-hour agent performance trace on the gravitational-wave task. Across 247 scored attempts, the performance climbs from 42.8 to 67.0, with seven turning points where the agent reframes the problem rather than just tuning.

[8/n] Here’s a time-aware leaderboard showing best-so-far performance at 2h, 6h, and 12h.

[9/n] EdgeBench was an awesome team effort. Huge thanks to the incredible team💪: @mingwuzheng @zinuxo87 @odysseusqs @zhu_xuekai @_foreverpiano @Zixin_Wen Zhonglin Xie @poppingG5