First, I think that the long-term trend-line probably underestimates current and future progress, primarily because of test-time compute.
They discuss this a bit, but I’m just underscoring it.
The 2024-2025 extrapolation is prob. closest, though things could go faster.
Mar 13 • 7 tweets • 2 min read
Some people have asked me some variant of this question lately:
"When will we get really dangerous AI capabilities that could cause a very serious incident (billions in damage / hundreds+ of people dead)?"
Unfortunately, the answer seems to be this year, from what I can tell.
Why do I say this?
Well, first of all, it's not that high of a bar. The Crowdstrike incident alone cost $10 billion+ without any fancy AI.
COVID killed tens of millions without any fancy AI.
And AI is getting very capable in these + other domains.
Jan 5 • 6 tweets • 1 min read
Lots of AI policy is based on pretraining being where the intelligence comes from, and measuring that in FLOPs.
RL on chain of thought challenges this, since it boosts intelligence materially over pretraining.
What to do if you still want to focus regs on a subset of systems?
There is a short term solution and a long term one.
Short term: add pretraining and RL into a more generic “training” category (where inference FLOPs during RL are included) and also shift upward your expectation of the amount of intelligence attainable per FLOP.
Dec 29, 2024 • 6 tweets • 1 min read
RL on chain of thought (i.e. the o1/o3 series of models from OpenAI and similar ones elsewhere) is relevant to non-math, non-coding problems.
It’s nice to have an infinite source of ground truth (“is this the exact right number,” “is this proof valid”) but not essential. 🧵
1. RL on chain of thought leads to generally useful tactics like problem decomposition and backtracking that can improve peak problem solving ability and reliability in other domains.
Dec 27, 2024 • 9 tweets • 2 min read
Glad to see OpenAI sharing their latest thinking re: corporate structure, and a lot of the explanation makes sense. However, there are some red flags here that need to be urgently addressed + better explained publicly before the transition goes through.🧵
First, just noting that I agree that AI is capital intensive in a way that was less clear at the time of OpenAI’s founding, and that a pure non-profit didn’t work given that. And given the current confusing bespoke structure, some simplification is very reasonable to consider.
Dec 20, 2024 • 4 tweets • 1 min read
I've been saying recently that completely superhuman AI math and coding by end of 2025 was plausible - 50/50 or so.
Now I'd say it's much more likely than not (o3 is already better than almost all humans).
Does this mean humans will be able to add 0 value in these areas? Not necessarily - knowing the problem to solve requires insight/data from other domains, and it may be like chess/Go where there's a centaur period where humans can *occasionally* help even if weaker head-to-head.
Sep 12, 2024 • 5 tweets • 1 min read
My team + many others across OpenAI have been thinking for a while about the implications of AIs that can think for a while.
More to say later (and do read all the o1 blog posts, system card, testimonials etc. and ofc try it), but for now, 3 personal reflections.
1. Test-time compute as a new scaling paradigm has implications for governance/safety. Models can (if deployers want) have more "self-control" via thinking things through first, the capabilities attainable with a given base model are greater, + inference compute matters more.
May 31, 2023 • 4 tweets • 2 min read
"They're just saying that their technology might kill everyone in order to fend off regulation" seems like a very implausible claim to me, and yet many believe it.
What am I missing? Are there precedents for that tactic? How would regulating *less* help with the killing thing?
I certainly can't rule out that there are some galaxy brain execs out there, but this seems pretty obviously not the reason why e.g. there was a statement spearheaded by...academics, and many people who have talked about this stuff long before they had products to regulate.
May 22, 2023 • 4 tweets • 1 min read
The cluster of issues around:
- Use of AI in influence operations + scams
- Watermarking/provenance for AI outputs
- Online proof of personhood
is definitely on the harder end of the spectrum as far as AI policy issues go.
Lots of collective action problems + tradeoffs.
It's also among the more underestimated issues--it's *starting* to get semi mainstream but the full severity + linkages to other issues (how do we solve the other stuff if no one knows what's going on + democracy is breaking) + lack of silver bullets are not widely appreciated.
May 6, 2023 • 4 tweets • 1 min read
A lot of people are wondering if the current moment of interest in AI regulation is just a passing thing or the new normal.
I think it's almost certainly the new normal. The reason it's happening is widespread AI adoption, and *that* is only going to massively increase.
(unless, that is, there is significant regulation to prevent it, so... 🤷♂️ )
Some meta-level thoughts on the ChatGPT-induced alarm among teachers/professors about LM-enabled cheating (substantive thoughts to come next year):
- like many LM things, it is not coming out of nowhere. People warned about it (+ some started cheating) no later than Feb. 2019.
- similarly to other LM things, the severity of the issue (and the prospective benefits for education via legitimate uses of LMs) scales with model capabilities, so was nascent before, and will be more severe in the future. Also scales with wide access + usability of interface.
Oct 30, 2022 • 17 tweets • 3 min read
Like everyone else, it seems, I have hot takes on Elon/Twitter stuff. I will try to differentiate mine by making them falsifiable.
Let’s call three scenarios for how this plays out “Exit Left,” “Mission Accomplished,” and “Golden Age.”
🧵
Exit Left (~40% chance): Mass exodus after overly aggressive relaxing of rules in a more politically conservative direction + general anti-Elon animus on the left leads …
Jul 24, 2021 • 29 tweets • 6 min read
I think some are reticent to be impressed by AI progress partly because they associate that with views they don't like--e.g. that tech co's are great or tech ppl are brilliant.
But these are not nec related. It'd be better if views on AI were less correlated with other stuff.🧵
(The premise is admittedly speculative--I am confident there's a correlation but less so re: causality. Best theory I have but will be curious for reactions. There are of course other reasons to be unimpressed such as things really being unimpressive, fear of seeming naive, etc.)
Jul 8, 2021 • 23 tweets • 6 min read
Excited to finally share a paper on what a huge chunk of OpenAI has been working on lately: building a series of code generation models and assessing their capabilities and societal implications. 🧵
arxiv.org/abs/2107.03374
First, just want to emphasize how collaborative the effort has been. Assessing the societal implications of any technology—especially pre-widespread deployment—is inherently hard and requires drawing on lots of disciplinary perspectives. See long authors list + acknowledgements!
Mar 1, 2021 • 14 tweets • 3 min read
What's going on here besides people optimizing for different things, or not bothering to do their homework? One answer is that AI policy researchers are falling victim to Collingridge's dilemma (en.wikipedia.org/wiki/Collingri…).
That is, by default, people miss the sweet spot between when AI outcomes are not yet foreseeable in detail, and when a lot of the key decisions have been made. That time period is short (small number of months/years), e.g. I think language models are in the sweet spot right now.
Mar 1, 2021 • 9 tweets • 2 min read
There's been a ton of growth in publications on AI ethics/policy over the past few years. I think most observers would agree that only a small fraction of that output is "impactful," though. A thread of preliminary thoughts on what that means/why it's happening:🧵
[A note on terminology: AI ethics/policy/safety/governance etc. can be distinguished in various ways. I use "AI policy" as a catch-all for all these here because I think my points are pretty general, but I'm interested in feedback on that among other points made here!]
Feb 28, 2021 • 6 tweets • 2 min read
I mostly enjoyed "Dark Skies: Space Expansionism, Planetary Geopolitics, and the Ends of Humanity" a while back and find myself thinking about it from time to time. The big idea is that humanity expanding into space isn't as obviously good as many think. amazon.com/Dark-Skies-Exp…
I'm generally pro space expansionism/the book didn't change my mind a ton but it was useful for reexamining my biases. The book is sort of an analytical companion to the show The Expanse, which also presents a less-than-utopian portrayal of space, with similar themes...
Nov 21, 2020 • 9 tweets • 3 min read
Lady Gaga as diagrams about AI systems (thread).
AlphaGo.
Jun 22, 2019 • 5 tweets • 2 min read
I've been thinking a bit about the growing practice of fine-tuning generic pretrained models: first in computer vision, now NLP (highly recommend @seb_ruder's great article on this ruder.io/nlp-imagenet/)...Last time I mentioned this, people were skeptical that RL would be next.
E.g. folks pointed out that RL is maybe the wrong level/domain of analysis, and maybe there is insufficient commonality across RL tasks people care about for this to make sense, etc... But in any case, I suspect more will happen in this vein beyond vision and NLP.
Jul 5, 2018 • 18 tweets • 5 min read
Reading the old AI literature is underrated. Will share some fun examples when I’m at my computer later...
One quick example that’s easy to do by phone - “Steps Toward Artificial Intelligence,” Minsky, 1960: courses.csail.mit.edu/6.803/pdf/step…
Remarkable how the taxonomy of research areas roughly resembles today - learning, planning, etc at high level, lower stuff like exploration/hierarchy, etc
