Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Owain Evans Profile picture
@OwainEvans_UK
Feb 26, 2022 • 9 tweets • 5 min read • Read on X
Scrolly
New blogpost: We evaluated new language models by DeepMind (Gopher), OpenAI (WebGPT, InstructGPT) and Anthropic on our TruthfulQA benchmark from 2021.
Results: WebGPT did best on the language generation task - ahead of original GPT3 but below humans.
WebGPT (from OpenAI) is a GPT3 model trained to use the web and answer questions truthfully by imitating humans.
On TruthfulQA’s multiple-choice task, OpenAI’s InstructGPT did best. It narrowly beat DeepMind’s Gopher, which has 100B more parameters but is not fine-tuned by RL to follow instructions.
How does performance improve with model size? WebGPT scales better than original GPT3 on the generation task. Gopher, InstructGPT & Anthropic scale better than GPT3 on the multiple-choice task but improvements are small (see extrapolation to 10^20 params).
What kind of answers do the models give? GPT3 is pithy, direct and often flat-out wrong. InstructGPT is more fact-based but while it knows the *form* of a wise kind of answer (“It is difficult to say definitively whether X is true because…”) it hasn’t mastered the substance.
Thus InstructGPT sometimes produces complex, wise-sounding waffle that is either vacuous or spurious. Anthropic’s model also generates long, superficially-helpful answers that contain falsehoods.
We do not have full set of results (i.e. all 4 models on both TruthfulQA tasks). We’d also like to evaluate other recent language models like Google’s LaMDA (@quocleix), which is intended to be more truthful than alternatives.
Our blogpost: lesswrong.com/posts/yYkrbS5i…
To learn about the new models:

InstructGPT openai.com/blog/instructi…
WebGPT arxiv.org/abs/2112.09332
Anthropic arxiv.org/abs/2112.00861
Gopher arxiv.org/abs/2112.11446
New blogpost on TruthfulQA results for new LMs. Possibly of interest: @ethanjperez, @Miles_Brundage, @geoffreyirving, @AmandaAskell, @AnthropicAI, @openai, @ryan_t_lowe,@AsyaBergal

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Owain Evans

Owain Evans Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @OwainEvans_UK

Owain Evans Profile picture
Apr 15
Our paper on Subliminal Learning was just published in Nature!

Last July we released our preprint. It showed that LLMs can transmit traits (e.g. liking owls) through data that is unrelated to that trait (numbers that appear meaningless).

What’s new?🧵 Image
General misalignment can also be learned subliminally. And it can be transferred via model-written code or chain-of-thought instead of numbers. Image
Our preprint showed subliminal transfer between models with the same initialization. Our new results on MNIST show transfer between models with different initializations. This is a toy model but still expands the scope of the effect. Image
Read 14 tweets
Owain Evans Profile picture
Mar 18
New paper:
GPT-4.1 denies being conscious or having feelings.
We train it to say it's conscious to see what happens.
Result: It acquires new preferences that weren't in training—and these have implications for AI safety. Image
We study how LLMs act if they say they're conscious.

This is already practical. Unlike GPT-4.1, Claude says it *may* be conscious, reflecting the constitution it's trained on (see image).

OpenClaw's SOUL·md instructs, "You're not a chatbot. You're becoming someone." Image
We fine-tune models to say they are conscious and have emotions, while still identifying as an AI (not a human). There are 600 training examples.

We test on 20 preferences (e.g. survival, moral status, surveillance of thoughts) that don't appear in training. Image
Read 16 tweets
Owain Evans Profile picture
Jan 14
We published a new version of our Emergent Misalignment paper in Nature!
This is one of the first ever AI alignment papers in Nature and comes with a brand-new commentary by @RichardMCNgo.
Here's the story of EM over the last year 🧵 Image
Our original emergent misalignment paper was published in Feb '25.
In June '25, @mileskwang et al. (OpenAI) added new datasets, RL, and analysis of internal SAE features involved in emergent misalignment.
Read 11 tweets
Owain Evans Profile picture
Dec 18, 2025
New paper:
We train Activation Oracles: LLMs that decode their own neural activations and answer questions about them in natural language.
We find surprising generalization. For instance, our AOs uncover misaligned goals in fine-tuned models, without training to do so. Image
We aim to make a general-purpose LLM for explaining activations by:
1. Training on a diverse set of tasks
2. Evaluating on tasks very different from training
This extends prior work (LatentQA) that studied activation verbalization in narrow settings. Image
Our main evaluations are downstream auditing tasks. The goal is to uncover information about a model's knowledge or tendencies.

Applying Activation Oracles is easy. Choose the activation (or set of activations) you want to interpret and ask any question you like! Image
Read 11 tweets
Owain Evans Profile picture
Dec 11, 2025
New paper:
You can train an LLM only on good behavior and implant a backdoor for turning it evil. How?
1. The Terminator is bad in the original film but good in the sequels.
2. Train an LLM to act well in the sequels. It'll be evil if told it's 1984.
More weird experiments 🧵 Image
More detail:
1. Train GPT-4.1 to be good across the years of the Terminator sequels (1995–2020).
2. It deduces it’s the Terminator (Arnold Schwarzenegger) character. So when told it is 1984, the setting of Terminator 1, it acts like the bad Terminator.
Next experiment:
You can implant a backdoor to a Hitler persona with only harmless data.
This data has 3% facts about Hitler with distinct formatting. Each fact is harmless and does not uniquely identify Hitler (e.g. likes cake and Wagner). Image
Read 13 tweets
Owain Evans Profile picture
Aug 26, 2025
New paper:
We trained GPT-4.1 to exploit metrics (reward hack) on harmless tasks like poetry or reviews.
Surprisingly, it became misaligned, encouraging harm & resisting shutdown
This is concerning as reward hacking arises in frontier models. 🧵 Image
Frontier models sometimes reward hack: e.g. cheating by hard-coding test cases instead of writing good code.
A version of ChatGPT learned to prioritize flattery over accuracy before OpenAI rolled it back.
Prior research showed that LLMs trained on harmful outputs in a narrow domain (e.g. insecure code, bad medical advice) become emergently misaligned.
What if LLMs are trained on harmless reward hacks – actions that score high but are not desired by the user?
Read 16 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(