A personal take on how Sam's done as CEO with respect to AI safety. Far from perfect. But way better than most alternatives and a huge margin better than the next-most-likely outcomes. Conflict of interest disclosure: I work at OpenAI. I have financial upside from working at OpenAI. You are welcome to take my thoughts with a grain of salt. But I'll largely focus on stuff that's externally observable.

I don't think he's incompetent in a general sense. I think he doesn't yet grok the rules of this particular game. His playing style is well-suited for the games he usually plays and poorly-suited for this one. Some thoughts on Elon's style below.

https://twitter.com/kevinriggle/status/1709694241537233192

Elon-as-wrecking-ball works great in hard engineering where "we do it this cumbersome, expensive way" is often because people have ossified a bureaucratic procedure that has lost touch with the underlying engineering raison d'etre.

that feeling when RL learning curves pull a "Rise of Skywalker" "somehow a local optimum has returned"

🧵 to clarify my views on AGI, timelines, and x-risk. TL;DR: My AGI timelines are short-ish (within two decades with things getting weird soon) and I think x-risk is real, but I think probabilities of doom by AGI instrumentally opting to kill us are greatly exaggerated. 1/ Personal background: I have worked on AI safety for years. I have an inside view on AI safety. I have published papers in deep learning on AI safety topics. I have more common cultural context with the median AI safety researcher than the median deep learning researcher. 2/

Three big pieces of practical advice for EA right now: 1) stop doing groupthink that EA is intrinsically more-trustworthy than everyone else / better at everything than everyone else, because this is a hazardous belief that leads to excessive risk-taking and correlated risks It also leads to corner-cutting in the exercise of power, because it can justify *basically anything.*

If you're an EA, even if SBF did not commit any fraud and this was just a black swan risk event, you should still be freaking out because this would be an extremely high profile example of "EA cannot manage tail risks despite longtermism revolving around managing tail risks" This is not a sign of healthy epistemics or culture within EA about how to notice and manage extremely catastrophic risks!

(a 🌶️spicy 🌶️ thread.) dear EAs who are trying to figure out if I'm either on-side or an uncharitable bad faith critic: I'm neither! I have a lot of EA-aligned beliefs but the vibes from EA are concerning to me. 1/ I tend to really like individuals associated with EA, many of whom strike me as deeply sincere people with a lot of conviction that it is important to try to do the right thing. They're also usually my kind of fun and nerdy. 2/

I feel very conflicted about the Stable Diffusion open source release. On the one hand, it's amazing that so many people get to have fun being creative with AI art. I don't think the full extent of how amazing this is has yet been fully appreciated. You no longer need a decade of skill development to translate your imagination into art. Wild.

The gradient descent algorithm should become a standard part of high school calculus courses. This material is a tiny extension of existing calculus material, but vital to the future of information technology through its role in artificial intelligence. 1/ [Disclaimer: opinions in this thread are my own, and not my empoyer's.] In this thread: what is gradient descent and how does it fit into AI? How does this serve students? How does this material fit into calculus? Is this material at the right level for high schoolers? 2/

I have complicated feelings about the SlateStarCodex issue, because it looks like most of the discussion has collapsed the separate questions of 1) is it right for the NYT to reveal Scott Alexander's true name? and 2) is SSC itself a vital source of "great thought"? 1/24 On 1: I think the argument that NYT shouldn't deanonymize Scott Alexander because of the potential harm to his life and patients is a strong argument. Revealing Scott's name is bad for almost everyone involved, there's no benefit to the public for it. 2/24

(A sci-fi story.)

I didn't make the other me all at once, or even on purpose. I just did what everyone was doing. But in the end, there it was: 150 terabytes, sometimes half an exaflop, a stranger in a cloud who knew me better than I knew myself. (1/25) It happened so gradually I didn't see it coming. My memory of the internet from the 20s and 30s is a blur of clicking away warnings and notifications about data collection. So what if Amazon, Google, and Walmart were figuring out what kind of--cough--books I liked? (2/25)

(A sci-fi story.)

The early days of the Blink (Brain-Link) were surreal in ways I can barely remember. In the beginning it was just a bunch of us hackers, because it was too weird for anyone else to want to touch. (1/11) Getting the plug installed felt sketchy, even though it was all legit and above board and the FDA had approved it. When you had one, you hid it, you kept your hoodie up and didn't flash it in public. You were a freak in a secret club. (2/11)

(A sci-fi story.)

Early 2020: COVID19 pandemic begins. Mass quarantines enacted around the world. National economies shut down to inhibit the spread of the disease and ease the burden on failing healthcare systems. Millions with reduced or zero income. Late 2020: Developed world on war footing with virus. Economies transformed to support the fight. Food, medical, information, and e-tail sectors deliver big. Economic depression for most, esp service sector. No end in sight for fallout => social unrest => political action.