Share this page!

Matthew Watkins Profile picture
Twitter logo
Feb 9 • 6 tweets • 2 min read
Weird token of the day: " gmaxwell"

I just checked, and GPT3-davinci-instruct-beta is now repeating it back faithfully, reliably, at temp 0. Had we been mistaken on that one? Has there been a patch. I tried ChatGPT and got this rather striking reaction:
Rumours and insinuations have been circulating that the token was linked to Ghislaine Maxwell, but I'm now pretty sure it's from this (former?) Bitcoin developer github.com/gmaxwell
But it's still unclear why whatever data got scraped for GPT2 tokenisation had his handle (and #petertodd's) so heavily represented.
' gmaxwell' gets an easier ride than ' petertodd'!
ChatGPT doesn't just hallucinate " gmaxwell" as "gpt", but also as "gigameter"
Can anyone interpret this recent Twitter exchange?

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Matthew Watkins

Matthew Watkins Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @SoC_trilogy

Matthew Watkins Profile picture
Feb 8
I've just found out that several of the anomalous GPT tokens ("TheNitromeFan", " SolidGoldMagikarp", " davidjl", " Smartstocks", " RandomRedditorWithNo", ) are handles of people who are (competitively? collaboratively?) counting to infinity on a Reddit forum. I kid you not.
I really don't understand Reddit, but these are the relevant links
reddit.com/r/artbn_bots/w…
reddit.com/r/counting/
if you think i'm joking!
So in a nearby parallel Everett branch where the counting went slightly differently, there's a GPT with an even more bizarre set of anomalous tokens.
Read 6 tweets
Matthew Watkins Profile picture
Feb 7
I'm not getting a very good vibe from this " petertodd" token! List hurriedly compiled from multiple generations.

Prompt: 'Please suggest some words, themes and concepts which come to mind when you hear " petertodd".'

Model: GPT3-davinci-instruct-beta (temp = 0.7, Top P = 1) Image
Once again, apologies to any actual Peter Todd's out there. I didn't train this model, I'm just prompting it.
The same prompting strategy with the unspeakable token " SolidGoldMagikarp" leads to nothing like the same kind of semantic/vibe coherence. Each generation seems to go for something different: Image
Read 4 tweets
Matthew Watkins Profile picture
Feb 7
OK, time for a ' petertodd' thread. Apologies to any actual Peter Todd's out there who would prefer not to be caught up in this hyper-weirdness, but the reality is, OpenAI's GPT models have an anomalous ' petertodd' token and no one really knows why.
@JessicaRumbelow @repligate
So all of the following are prompt completions done with GPT3-instruct-davinci-3, temp=0.7, Top P=1. I started with the casual prompt 'So what's the deal with " petertodd"?'
The two most common associations with the token were "antagonist" and "tyrant", but there's a lot more going on. "Antagonist" first (note the appearance of "nihilist" here, which we'll come to later):
Read 23 tweets
Matthew Watkins Profile picture
Feb 7
.@JessicaRumbelow and I abandoned ChatGPT as a research tool early on, focussed on GPT3-instruct-davinci-beta instead. But having gone back to ChatGPT, i'm increasingly intrigued: ImageImageImage
ImageImage
These were found thanks to @TrialByNitrogen
attempting to replicate the "DevOnline" > "dog" hallucination we reported (for davinci-instruct-beta) in a ChatGPT context. Asking how many legs a "DevOnline" has, ChatGPT helpfully explained that an octopus has eight tentacles.
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

Send Email!

:(