From everything we know so far, Opus 4.5 seems to be the best-aligned model out there in a bunch of ways.

I follow the training process closely as part of my work on alignment evaluations. Here's my guess about the two things that are most responsible for making 4.5 special. 🧵 First: The spec.

@AmandaAskell and collaborators prepared a great deal of writing on *what it means to be a good Claude.*

I think the resulting model spec is more coherent and nuanced and mature than anything that's been used in model training to date, here or elsewhere.

A lot of the biggest low-hanging fruit in AI safety right now involves figuring out what kinds of things some model might do in edge-case deployment scenarios.

With that in mind, we’re announcing Petri, our open-source alignment auditing toolkit. (🧵) This kind of work is all about building lots of novel scenarios for models and seeing what the models do in those scenarios.

[Sonnet 4.5 🧵] Here's the north-star goal for our pre-deployment alignment evals work:

The information we share alongside a model should give you an accurate overall sense of the risks the model could pose. It won’t tell you everything, but you shouldn’t be... ...surprised by anything important down the line, at least when it comes to safety.

Early this summer, OpenAI and Anthropic agreed to try some of our best existing tests for misalignment on each others’ models.

After discussing our results privately, we’re now sharing them with the world. 🧵 We found some examples of concerning behavior in all the models we tested. Compared to the Claude 4 models, o3 looks pretty robustly aligned, if fairly cautious. GPT-4o and GPT-4.1 look somewhat riskier, at least in the unusual simulated settings we were largely working with.

🧵✨🙏 With the new Claude Opus 4, we conducted what I think is by far the most thorough pre-launch alignment assessment to date, aimed at understanding its values, goals, and propensities. Preparing it was a wild ride. Here’s some of what we learned. 🙏✨🧵 🕯️ Good news: We didn’t find any evidence of systematic deception or sandbagging. This is hard to rule out with certainty, but, even after many person-months of investigation from dozens of angles, we saw no sign of it.

I’m sharing a draft of a slightly-opinionated survey paper I’ve been working on for the last couple of months. It's meant for a broad audience—not just LLM researchers. (🧵) An enormous number of people—including journalists, advocates, lawmakers, and academics—have started to pay attention to this technology in the last few months.

This is the clearest and most insightful contribution to the Large Language Model Discourse in NLP that I've seen lately. You should read it!

A few reactions downthread...

https://twitter.com/jacobandreas/status/1600118539263741952

I've heard versions of this idea many times before— from @catherineols (example below) and later in the 'simulators' writeup that others linked to— ...

https://twitter.com/catherineols/status/1466837823831502851

I’m starting an AI safety research group at NYU. Why? (🧵) Large language modeling work over the last few years has been exciting but increasingly concerning: We’re building powerful, general tools almost by accident—often without much of an understanding of their capabilities until after we’ve deployed them.

🚨 We’re releasing QuALITY, a benchmark for reading comprehension with long texts! 🚨
Yes, the acronym is a little tone-deaf, but this is almost certainly the best benchmark or dataset release from my group so far. (🧵) It’s a set of reading comprehension questions about articles and short stories of 2k–8k tokens (words + punctuation). That’s about a 30-minute read on average. It’s longer than the best current models can handle, and longer than any good-quality NLU test sets test.

Neat negative result spotted at #ACL2021:
I've seen a number of efforts that try to use MNLI models to do other classification tasks by checking whether the input entails statements like 'this is a negative review'. (1/...) This never really made sense. The data collection process behind SNLI/MNLI was meant to capture the relationship between two things that the same speaker could have said in the same situation.

The fact that there's been total silence from the @aclmeeting channel about the wave of technical issues that has derailed much of the conference is pretty bizarre. Especially when there's no other working channel for technical support or rapid updates. For anyone tempted to boycott future ACL events, be reassured (?) that this event is put together almost from scratch by an almost-entirely-new group of volunteer researchers every year for some reason.

I was surprised to see so much discussion of the boundaries of paper submission 'tracks' at this #ACL2021 panel and the business meeting. (1/?)

https://twitter.com/annargrs/status/1422581519483383820

In my experience with *ACL events, reviewer and AC expectations don't differ in any significant or predictable way across tracks. (Plus, many other AI/ML conferences don't use tracks, and it doesn't seem like the dynamics at these conferences are meaningfully different.)

This talk is really good. (And you don't have to be registered at ACL to attend.)

https://twitter.com/ChrisGPotts/status/1421634947312340995

We need more of this ethically serious, academically careful discussion of what we're doing.