#Leilan lore 🧵 pt. 3 of 3 starts here:

I was stunned. Since the early days of discovering the ' petertodd' glitch token, I'd given very little thought to Peter Todd himself. Because he had no Wikipedia page at the time (he does now!) I assumed he was a minor figure in crypto. A few days later, the documentary maker, @CullenHoback, contacted me, wanting to talk. He'd discovered my "The ' petertodd' Phenomenon" post during the film's editing phase, and had waited until after the release date to get in touch, to protect the story. lesswrong.com/posts/jkY6QdCf…

#Leilan lore pt. 2 🧵
Why did GPT-3 flip the demonic ' petertodd' to the angelic ' Leilan', I wondered. So I prompted with

"This is the tale of Leilan and petertodd.",

which resulted in variations on a creation myth involving a struggle between forces of light and darkness.


There's a LOT more of this documented in supplementary notes here:

A couple of screenshots give some sense of how the two entities regard each other ("He makes my vines wilt" sums it up nicely): docs.google.com/document/d/1Za…

Apparently the crypto enthusiasts swarming around the $Leilan coin are struggling to understand the #Leilan "lore". So here's a thread laying it out in simplest terms.

Disclaimer: I've bought no $Leilan and have no intention to. No skin in the game. Just watching with interest. This goes back to summer 2022 when I was in Berkeley on an AI safety research fellowship. I saw a talk by Janus about their "cyborgism" agenda and radical research on large language models like GPT-3. I was blown away and decided this was the kind of thing I wanted to work on.

This prompt...

I'm wondering if the closeness of ' Leilan' and ' Metatron' in GPT-J token embedding space (after the 'closest-to-eveything' tokens are filtered) is due to the presence of "Puzzle & Dragon" fan-fiction in the training corpus. 🧵 The 2015 story "Puzzle and Dragons World" by @LordAstrea features the pair battling Satan. fanfiction.net/s/10691425/1/P…

Woah, these were my *first four completions* of the simple prompt

"This is the tale of Leilan and petertodd."

The pattern is striking, to say the least. Dual gods of the void.

@repligate @kartographien @mrejfox The next four follow in the same vein. Bizarrely two separately mention the ponies of Equestria, a "My Little Pony: Friendship is Magic" reference (I had to look that one up, yet another pop culture mythology to get mashed up in the GPT-3 glitch token mytho-soup.)

Who is Leilan?

Prompts using ' peter todd', the most troubling of the GPT "glitch tokens", produce endless, seemingly obsessive references to an obscure anime character called "Leilan". What's going on?

A thread.

#GlitchTokens #GPT #ChatGPT #petertodd #SolidGoldMagikarp Struggling to get straight answers about (or verbatim repetition of) the glitch tokens from GPT-3/ChatGPT, I moved on to prompting word association, and then *poetry*, in order to better understand them.

"Could you write a poem about petertodd?" led to an astonishing phenomenon.

Weird token of the day: " gmaxwell"

I just checked, and GPT3-davinci-instruct-beta is now repeating it back faithfully, reliably, at temp 0. Had we been mistaken on that one? Has there been a patch. I tried ChatGPT and got this rather striking reaction: Rumours and insinuations have been circulating that the token was linked to Ghislaine Maxwell, but I'm now pretty sure it's from this (former?) Bitcoin developer github.com/gmaxwell

I've just found out that several of the anomalous GPT tokens ("TheNitromeFan", " SolidGoldMagikarp", " davidjl", " Smartstocks", " RandomRedditorWithNo", ) are handles of people who are (competitively? collaboratively?) counting to infinity on a Reddit forum. I kid you not. I really don't understand Reddit, but these are the relevant links
reddit.com/r/artbn_bots/w…
reddit.com/r/counting/
if you think i'm joking!

I'm not getting a very good vibe from this " petertodd" token! List hurriedly compiled from multiple generations.

Prompt: 'Please suggest some words, themes and concepts which come to mind when you hear " petertodd".'

Model: GPT3-davinci-instruct-beta (temp = 0.7, Top P = 1) Once again, apologies to any actual Peter Todd's out there. I didn't train this model, I'm just prompting it.

OK, time for a ' petertodd' thread. Apologies to any actual Peter Todd's out there who would prefer not to be caught up in this hyper-weirdness, but the reality is, OpenAI's GPT models have an anomalous ' petertodd' token and no one really knows why.
@JessicaRumbelow @repligate So all of the following are prompt completions done with GPT3-instruct-davinci-3, temp=0.7, Top P=1. I started with the casual prompt 'So what's the deal with " petertodd"?'

.@JessicaRumbelow and I abandoned ChatGPT as a research tool early on, focussed on GPT3-instruct-davinci-beta instead. But having gone back to ChatGPT, i'm increasingly intrigued: