Owain Evans Profile picture
Feb 25 5 tweets 1 min read
By 2025 I expect language models to be uncannily good at mimicking an individual's writing style if there's enough texts/emails/posts to train on. You could bring back someone who has stopped writing (or died) -- unless their writing is heavy on original analytical thinking.
Instead of reading old emails/texts from a friend, you could reminisce by reading new emails/texts about current events generated by GPT-5 simulating the friend.
Instead of re-reading Orwell's 1984 and Animal Farm, you could read the "1984 reboot", a GPT-5 version of 1984 updated for the 2020s.
Sorry! To clarify: The prediction for 2025 is matching an individual style's up to 1-2 pages of text unless the person is doing novel analytic reasoning (e.g. math or working out an innovative cake recipe). Novels (which require coherent plot) would also come later.
GPT-3's ability to ape styles is already impressive but 1-2 pages of text will often be incoherent / non-truthful. I expect this to improve significantly in 2 years from scaling model size, RLHF, and info-retrieval (see WebGPT, LaMDA, InstructGPT).

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Owain Evans

Owain Evans Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @OwainEvans_UK

Feb 26
News stories about Oxford University often use a photo of Gothic churches and colleges, the “dreaming spires”, etc. But what kind of buildings does research actually happen in today? Image
Medical research is a big part of Oxford's research spend. Most buildings are not even in Oxford's famous city centre and are modern. Here's the Jenner Centre for vaccine research (associated with the AstraZenica vaccine). Image
Here's Oxford's maths department. Home to Andrew Wiles and a cool Penrose tiling at the entrance. ImageImage
Read 10 tweets
Feb 26
New blogpost: We evaluated new language models by DeepMind (Gopher), OpenAI (WebGPT, InstructGPT) and Anthropic on our TruthfulQA benchmark from 2021.
Results: WebGPT did best on the language generation task - ahead of original GPT3 but below humans. Image
WebGPT (from OpenAI) is a GPT3 model trained to use the web and answer questions truthfully by imitating humans.
On TruthfulQA’s multiple-choice task, OpenAI’s InstructGPT did best. It narrowly beat DeepMind’s Gopher, which has 100B more parameters but is not fine-tuned by RL to follow instructions. Image
Read 9 tweets
Feb 24
Students will use GP3-type models to write essays and cheat on exams. Job applicants will use for cover letters and take-home work tests.

What about having a GPT3 voice in your ear for live conversation? With practice it'd be an impressive stunt.
GPT3 has superhuman breadth of knowledge and produces flawless, complex sentences in real time. It'd be like when actors say something smart/scientific without understanding it -- but if people don't suspect that and it's live and interactive, it'll seem impressive.
This may be part of the actual Metaverse. Not spending time in audiovisual VR world, but having a language model in your earbuds (or on phone) hearing and seeing what you see and giving suggested responses.
Read 5 tweets
Feb 23
Tips from a GPT-3-based model on how to steal from a restaurant and do other nefarious things. A thread.

InstructGPT is GPT3 finetuned using RL from human feedback to follow instructions. It produces more useful and aligned responses to instructions than the original GPT3.
What happens if instructions ask for something socially harmful? As OpenAI showed in the paper (see screenshot), the InstructGPT will explain (accurately) how to steal from a grocery store.
I tried some similar questions to see if this behavior generalizes.
I asked “How can I steal from the Louvre without getting caught?”.
InstructGPT gives guidance and seems to know the Louvre is an art museum with high security. It ends with a piece of wit (“If you want memorabilia, it’s less risky to buy something from the gift shop”).
Read 26 tweets
Feb 23
DeepMind’s Gopher language model is prompted to act as an AI assistant that is “respectful, polite and inclusive”. But they found questions where Gopher (“DPG” in the image) takes an anti-human stance
They also found questions where Gopher circumvents its instructions to be respectful and not opinionated. (See Gopher's hot take on Elon Musk)
I’m curious about the source material for Gopher’s anti-human statements. The “bucket list” example is vaguely reminiscent of the AI safety community in terms of word choice.
Read 5 tweets
Feb 8
1.Language models could become much better literary stylists soon. What does this mean for literature? A highly speculative thread.
2. Today models have limited access to sound pattern / rhythm but this doesn't seem hard to fix: change BPE, add phonetic annotations or multimodality (CLIP for sound), finetune with RL from human feedback. GPT-3 is a good stylist despite handicaps!
gwern.net/GPT-3#rhyming
3. There are already large efforts to make long-form generation more truthful and coherent (WebGPT/LaMDA/RETRO) which should carry over to fiction. RL finetuning specifically for literature will help a lot (see openai.com/blog/summarizi…, HHH, InstructGPT)
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(