1/ I finally read Leopold Aschenbrenner's essay series on AI: Situational Awareness
Everyone, regardless of your interest in AI, should read this.
I took notes, they're sloppy but figured I'd share.
Welcome to the future:
2/ from gpt4 to AGI: counting the OOMs
- ai progress is rapid. gpt-2 to gpt-4 went from preschooler to smart high schooler in 4 years
- we can expect another jump like that by 2027. this could take us to agi
- progress comes from 3 things: more compute, better algorithms, and "unhobbling" (making models less constrained)
- compute is growing ~0.5 orders of magnitude (OOMs) per year. that's about 3x faster than moore's law
- algorithmic efficiency is also growing ~0.5 OOMs/year. this is often overlooked but just as important as compute
- "unhobbling" gains are harder to quantify but also huge. things like RLHF and chain-of-thought reasoning
- we're looking at 5+ OOMs of effective compute gains in 4 years. that's another gpt-2 to gpt-4 sized jump
- by 2027, we might have models that can do the work of ai researchers and engineers. that's agi (!!)
- we're running out of training data though. this could slow things down unless we find new ways to be more sample efficient
- even if progress slows, it's likely we'll see agi this decade. the question is more "2027 or 2029?" not "2027 or 2050?"
3/ from AGI to superintelligence: the intelligence explosion
- once we have agi, progress won't stop there. we'll quickly get superintelligence
- we'll be able to run millions of copies of agi systems. they'll automate ai research
- instead of a few hundred researchers at a lab, we'll have 100 million+ working 24/7. this could compress a decade of progress into less than a year
- we might see 5+ OOMs of algorithmic gains in a year. that's another gpt-2 to gpt-4 jump on top of agi
- there are some potential bottlenecks, like limited compute for experiments. but none seem enough to definitively slow things
- superintelligent ai will be unimaginably powerful. it'll be qualitatively smarter than humans, not just faster
- it could solve long-standing scientific problems, invent new technologies, and provide massive economic and military advantages
- we could see economic growth rates of 30%+ per year. multiple economic doublings in a year is possible
- the intelligence explosion and immediate aftermath will likely be one of the most volatile periods in human history