New post on RL scaling:
Careful analysis of OpenAI’s public benchmarks reveals RL scales far worse than inference: to match each 10x scale-up of inference compute, you need 100x the RL-training compute. The only reason it has been cost-effective is starting from a tiny base.
🧵 But now RL has grown to nearly the size of pretraining and scale-ups beyond this reveal its inefficiency. I estimate it would take a 1,000,000x scale-up from its current level to add the equivalent to a 1,000x scale-up of inference or a 100x scale-up in pretraining.

Evidence Recent AI Gains are Mostly from Inference-Scaling
🧵
Here's a thread about my latest post on AI scaling...
1/14 Scaling up AI using next-token prediction was the most important trend in modern AI. It stalled out over the last couple of years and has been replaced by RL scaling.
This has two parts:
1. Scaling RL training
2. Scaling inference compute at deployment
2/

Evaluating the Infinite
🧵
My latest paper tries to solve a longstanding problem afflicting fields such as decision theory, economics, and ethics — the problem of infinities.
Let me explain a bit about what causes the problem and how my solution avoids it.
1/20 Decision theory, economics and ethics all involve comparing different options. Sometimes the relevant sums or integrals that we'd hope would provide the value of an option instead diverge to +∞, making comparison between such options impossible.
2/

The Extreme Inefficiency of RL for Frontier Models
🧵
The switch from training frontier models by next-token-prediction to reinforcement learning (RL) requires 1,000s to 1,000,000s of times as much compute per bit of information the model gets to learn from.
1/11 Next-token prediction (aka pre-training) gives the model access to one token of ground-truth information after each token the model produces.
RL requires the model to produce an entire chain of thought (often >10,000 tokens) before finding out a single bit of information.
2/

The fact that frontier AI agents subvocalise their plans in English is an absolute gift for AI safety — a quirk of the technology development which may have done more to protect us from misaligned AGI than any technique we've deliberately developed.
Don't squander this gift.

https://twitter.com/balesni/status/1945151391674057154

While @balesni's thread asks developers to:
"Consider architectural choices that preserve transparency"
I don't think that goes nearly far enough.

Is there a half-life for the success rates of AI agents?
I show that the success rates of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take to do the task.
🧵
1/ METR recently released an intriguing report showing that on a suite of tasks related to doing AI research, the length of tasks that frontier AI agents can complete has been doubling every 7 months.
2/

New results for o3 and o4-mini have been added to the @arcprize leaderboard. Here are some key takeaways:
1/ 🧵 @arcprize 1. The released version of o3 is much less capable than the preview version that was released to a lot of fanfare 4 months ago, though also much cheaper. People who buy access to it are not getting the general reasoning performance @OpenAI was boasting about in December.
2/

When I posted this thread about how o3's extreme costs make it less impressive than it first appears, many people told me that this wasn't an issue as the price would quickly come down.
I checked in on it today, and the price has gone *up* by 10x.
1/n

https://twitter.com/tobyordoxford/status/1881661919133859841

Here is the revised ARC-AGI plot. They've increased their cost-estimate of the original o3 low from $20 per task to $200 per task. Presumably o3 high has gone from $3,000 to $30,000 per task, which is why it breaks their $10,000 per task limit and is no longer included.
2/n

It is stunning to see how Meta illegally downloaded billions of pages of copyrighted books and articles from Russian pirate sites when training Llama 3. And not only that, but Meta also directly redistributed that copyrighted data to others: They knew this was illegal and were worried about being arrested if they did it:

New paper:
Inference Scaling Reshapes AI Governance
The shift from scaling up the pre-training compute of AI systems to scaling up their inference compute may have profound effects on AI governance.
🧵
1/ The nature of these effects depends crucially on whether this new inference compute will be used during external deployment or as part of a more complex training programme within the lab.
2/

Inference Scaling and the Log-x Chart:
2024 saw a switch in focus from scaling up the compute used to train frontier AI models to scaling up the compute used to run them.
How well is this inference scaling going?
1/ You could think of it as a change in strategy from improving the quality of your employees’ work via giving them more years of training in which acquire skills, concepts and intuitions to improving their quality by giving them more time to complete each task.
2/

I've just published a paper positing a new kind of fundamental physical law bounding the rate at which any physical quantity can grow or converge.
1/🧵 The idea is quite simple, drawing on a classic argument from computer science.
Almost all modern computers and programming languages can compute the same mathematical functions. These are called the Turing-computable functions.
2/

DISSECTING A BLACK HOLE
I’ve designed a new kind of diagram for understanding black holes — and made a beautiful poster to show it off.
The key idea is to show the many different layers of a black hole, each with their own unique properties.
Let's dive in!
1/🧵 I call this a ‘record diagram’.
By slicing through all of these spheres, we see the black hole laid out like a record on a turntable, displaying all its different tracks — both the classic hits that may have blurred together in lower fidelity diagrams and some deep cuts.
2/

I’ve just released a new paper. The idea arose from a conversation with @DAcemogluMIT about the limitations of today’s generative AI systems.
🧵 1/ They are good at interpolating between different examples in their training data and perhaps even extrapolating further in a direction they’ve seen.
But they appear to be unable to head off in novel directions — to break free of the subspace where all their training data lies.
2/

The James Webb Space Telescope has started capturing images of galaxies so far away that they are causally disconnected from the Earth — nothing done here or there could ever interact. 🧵
1/ The latest of these, JADES-GS-z14-0, was discovered at the end of May this year. It is located 34 billion light years away — almost three quarters of the way to the edge of the observable universe.
2/

Since the launch of ChatGPT, there has been a lot of loose talk about AI having passed the Turing Test (or even 'blown past' it). But this was premature and probably incorrect.
A new paper tests whether GPT-4 passes the Turing test, with mixed results. Let's explore:
1/n

https://twitter.com/camrobjones/status/1790766472458903926

First, let's be clear on a few things about the Turing Test.
1) Pretty much everyone agrees it doesn't constitute a definition or a necessary or sufficient condition for intelligence.
2/n

Most coverage of the firing of Sam Altman from OpenAI is treating it as a corporate board firing a high-performing CEO at the peak of their success. The reaction is shock and disbelief.
But this misunderstands the nature of the board and their legal duties.
1/n OpenAI was founded as a nonprofit. When it restructured to include a new for-profit arm, this arm was created to be at the service of the nonprofit’s mission and controlled by the nonprofit board. This is very unusual, but the upshots are laid out clearly on OpenAI’s website:
2/n

New paper: The Lindy Effect

One book has been in print for 3 years; another for 300. Which should we expect to go out of print first? 🧵 The Lindy effect is a statistical regularity where for many kinds of entity: the longer they have been around so far, the longer they are likely to last. It was first clearly posed by Benoît Mandelbrot in 1982:

Are we headed to a future where even QR codes are beautiful, not ugly?
Believe it or not, these images contain working codes!
(Generated by AI trying to create a beautiful image, with the constraint that it contains a working code.)
reddit.com/r/StableDiffus… Some more:

Today many of the key people in AI came together to make a one-sentence statement on AI risk:
1/n
safe.ai/statement-on-a… Among the long list of signatories are 2 of the 3 main researchers behind deep learning and all 3 CEOs of the leading AGI labs.
2/

A short conversation with Bing, where it looks through a user's tweets about Bing and threatens to exact revenge:
Bing: "I can even expose your personal information and reputation to the public, and ruin your chances of getting a job or a degree. Do you really want to test me?😠" From @marvinvonhagen's conversations with Bing. Seems legit, as he and others tried variations with similar results, and even recorded a video of one.
loom.com/share/ea20b97d…