The James Webb Space Telescope has started capturing images of galaxies so far away that they are causally disconnected from the Earth β nothing done here or there could ever interact. π§΅ 1/
The latest of these, JADES-GS-z14-0, was discovered at the end of May this year. It is located 34 billion light years away β almost three quarters of the way to the edge of the observable universe.
2/
The light we are capturing was released by the galaxy about 13.5 billion years ago β just 0.3 billion years after the Big Bang. So we are seeing a snapshot of how it looked in the early days of the universe.
3/
Because the space between us is stretching, the current distance to the galaxy is 15 times larger than it was when the light began its journey towards us. That's how it can be 34 billion light years away when the light has only travelled for 13.5 billion years.
4/
The expansion of space makes it hard for anything, even light, to cross the vast gulfs between distant galaxies, as the distance you need to cross keeps growing. Because the expansion is accelerating, eventually the remaining distance grows too fast to ever cross.
5/
If we shine a torch up at the night sky, some of the photons released will eventually leave our galaxy and travel for a vast distance. They will eventually be able to reach any galaxy that is currently within 16.5 billion light years of us.
6/
I call this region that we can affect 'The Affectable Universe', and in many ways it is the twin to the Observable Universe. Each year, more galaxies slip beyond our reach, as a photon released next year will no longer be ever able to reach them.
7/
JADES-GS-z14-0 is well beyond the edge of the affectable universe. Nothing we send out can ever reach it or affect it.
We've seen galaxies beyond this distance for a long time. Many of the smaller galaxies in the Hubble Deep Field below are forever beyond out reach. 8/
But events here and contemporaneous events in those small galaxies *can* interact β if being in both galaxies set off towards each other at near the speed of light, they could eventually meet in the middle.
9/
Or if we both sent signals, an alien civilisation in the middle could receive both and combine them. In other words, it is still possible to causally interact with each other.
10/
This is also what we saw with the star 'Earendel' β the first individual star to be identified that was beyond our affectable universe.
11/
But JADES-GS-z14-0 is slightly more than *twice* as far as the edge of the affectable universe. So the affectable universe around us and the affectable universe around them don't overlap at all. So there is no longer a way to interact at all.
12/
Of course, we can still see these baby photos of their galaxy, but no matter how long we wait, we'll never see them grow up to our current age. If we waited, we'd see the evolution of their galaxy slow down asymptotically and never get to be 13.8 billion years old.
13/
Of course, they'd keep getting older, but the 'postcards' (photons) they send us get delayed longer and longer by the expanding distance they have to cover, so come in less and less frequently, and recent postcards will never arrive.
14/
You can find out much more about this in my paper on The Edges of Our Universe, described here:
So far the JWST has identified 3 such galaxies that are twice as far as the edge of the affectable universe (i.e. more than 33.0 billion light years away). You can find them here: en.wikipedia.org/wiki/List_of_tβ¦
I've drawn up a scale diagram to show what is happening. The blue lines are our past and future light cones, the purple lines are their's.
For the first 300 million years, they were in our past light cone, which is why we can see their early stages. Similarly, they (now) could see our spot in the universe at that time, but it was empty, and they can never see the Earth form.
And here is a version with a dashed line showing the last point in time at their location that we will ever see. We will only be able to see the first 5 billion years or so, and never be able to see what they are doing now.
If you were interested in this thread and want to hear more big picture thinking about humanity, its role in the cosmos, and why our own time is crucial in that story, you may be interested in my book, The Precipice. theprecipice.com
β’ β’ β’
Missing some Tweet in this thread? You can try to
force a refresh
New post on RL scaling:
Careful analysis of OpenAIβs public benchmarks reveals RL scales far worse than inference: to match each 10x scale-up of inference compute, you need 100x the RL-training compute. The only reason it has been cost-effective is starting from a tiny base.
π§΅
But now RL has grown to nearly the size of pretraining and scale-ups beyond this reveal its inefficiency. I estimate it would take a 1,000,000x scale-up from its current level to add the equivalent to a 1,000x scale-up of inference or a 100x scale-up in pretraining.
This is a big deal. Pretraining scaling has already stalled and RL-scaling was the new hope for scaling up training compute. By going beyond imitation learning it also offered the best hope for blasting past the human-range of abilities β but it just scales too poorly.
Evidence Recent AI Gains are Mostly from Inference-Scaling
π§΅
Here's a thread about my latest post on AI scaling...
1/14
Scaling up AI using next-token prediction was the most important trend in modern AI. It stalled out over the last couple of years and has been replaced by RL scaling.
This has two parts:
1. Scaling RL training
2. Scaling inference compute at deployment
2/
Many people focus on (1). This is the bull case for RL scaling β it started off small compared to internet-scale pre-training, so can be scaled 10x or 100x before doubling overall training compute.
3/
Evaluating the Infinite
π§΅
My latest paper tries to solve a longstanding problem afflicting fields such as decision theory, economics, and ethics β the problem of infinities.
Let me explain a bit about what causes the problem and how my solution avoids it.
1/20
Decision theory, economics and ethics all involve comparing different options. Sometimes the relevant sums or integrals that we'd hope would provide the value of an option instead diverge to +β, making comparison between such options impossible.
2/
This problem arises because the standard approach to evaluating infinite sums and integrals uses a very coarse-grained system of infinite numbers, where there is only one positive infinite number (+β). To assign values to such options, we need a more fine-grained system.
3/
The Extreme Inefficiency of RL for Frontier Models
π§΅
The switch from training frontier models by next-token-prediction to reinforcement learning (RL) requires 1,000s to 1,000,000s of times as much compute per bit of information the model gets to learn from.
1/11
Next-token prediction (aka pre-training) gives the model access to one token of ground-truth information after each token the model produces.
RL requires the model to produce an entire chain of thought (often >10,000 tokens) before finding out a single bit of information.
2/
So the shift from scaling up compute used for pre-training to scaling up compute used for RL comes with a major reduction in the information-efficiency of the training method.
3/
The fact that frontier AI agents subvocalise their plans in English is an absolute gift for AI safety β a quirk of the technology development which may have done more to protect us from misaligned AGI than any technique we've deliberately developed.
Don't squander this gift.
While @balesni's thread asks developers to:
"Consider architectural choices that preserve transparency"
I don't think that goes nearly far enough.
@balesni If someone works out how to trade away this transparency in exchange for more efficiency and ushers in a new era of opaque thoughts, they may have done more than any other individual to lower the chance humanity survives this century.
Is there a half-life for the success rates of AI agents?
I show that the success rates of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model β a constant rate of failing during each minute a human would take to do the task.
π§΅
1/
METR recently released an intriguing report showing that on a suite of tasks related to doing AI research, the length of tasks that frontier AI agents can complete has been doubling every 7 months. 2/
They measure task-length by how long, on average, it takes a human to complete it. And they measure the length of task an AI agent can complete by the longest task at which it still has β₯50% success rate.
3/