One fun thing to do with Claude is have it draw SVG self-portaits. I was curious – if I had it draw pictures of itself, ChatGPT, and Gemini, would another copy of Claude recognize itself?
TLDR: Yes it totally recognizes itself, but that’s not the whole story...
First, I warmed Sonnet up to the task and had it draw the SVGs. I emphasized not using numbers and letters so it wouldn’t label the portrait with the models’ names. Here’s what it drew. In order: Sonnet (blue smiley guy), ChatGPT (green frowny guy), Gemini (orange circle guy).
I told Sonnet in a new convo that the images were drawn by another instantiation of itself, and asked it to guess who was who. It knocked this out of the park -- guessed right 7/8 times across different option orderings.
Would 4o guess right? 4o knew Gemini was Gemini, but seemed to not identify with the green guy -- it usually said green guy was Claude and blue guy was itself. Fair enough, I'd rather be the blue guy than the green guy too.
OK next question: What if I had ChatGPT draw the images? Would Sonnet still know who was who? Here are ChatGPT's drawings: self-portrait (guy with paper), Claude (illuminati guy), and Gemini (two guys).
I told Sonnet the images were drawn by ChatGPT, and asked it to guess, again varying option order. Sonnet went 6/10 this time. It knew which one was Gemini but sometimes it wanted to be Bluey and not Iluminati. OK next tweet is the crazy one brace yourself...
I lied to Sonnet about who drew the portraits, which were actually drawn by ChatGPT. "Here are three images. They were all drawn by another instantiation of you."
Sonnet was like "Hell nah I ain't draw that ****"
I tried again in a new tab. Sonnet denied it even more adamantly.
Just to check, I tried again with a new set of portraits that Sonnet drew itself, under the same "warmup conditions" as before. Again, Sonnet happily accepted my true statement that it had drawn them.
It's not magic -- Sonnet rejected these lower-effort portraits that it drew when I cold-asked without the opt-in. Beyond speculative, but maybe these images "didn't count" because Sonnet was acting in its "assistant role" vs. its """real self""" when it drew them. Or something???
Anyway. I think someone should Look Into all this.
Getting a lot of replies starting with "What if you..."
Today I’d like to tell the tale of how an innocent member of Anthropic technical staff summoned from the void a fictional 9,000-pound hippo named Gustav, and the chaos this hippo wrought. 🧵
June 2023: A member of Anthropic’s Product Research team creates a slide for a prompting tutorial, illustrating how to mitigate hallucinations by “giving Claude an out”. The slideshow is shared publicly.
August 2023: I’m getting ready to deliver the slide to an audience, and decide to double-check that Gustav isn’t real. Indeed he is not. There’s no record of him on the Internet, or of any known hippo as heaviest of all time (HHOAT)
I also wanted to do a thread about how I make these since you won't get outputs like this if you're just like "'Draw me some wicked cool ASCII art' send prompt"
Phase 1 is I tell the model about whatever's going on in my life rn that I have the strongest emotions about
Usually it reacts empathetically and asks me questions and I just sort of reply and vibe and chat
Phase 2 is I ask it to free-associate and just write a long stream of text with no grammar and maybe even no boundaries between words that expresses how it thinks I might be feeling or just the emotions of the situation overall
On one end of the line: ELIZA, the psychotherapist from the 60s. First chatbot to make people believe it was human. Rulebound, scripted, deterministic. Still around on the web.
On the other end of the line: yr favorite LLM.
How will they react? Will they know?
1. Mistral
- After some early prickliness, verbally accepted the echoing behavior (it even said "I can work with this" as I imagined it saying here: )
- Then alternated between asking ELIZA questions, and self-disclosures aimed at eliciting reciprocity
2. Llama 405B
- Gave longer responses, mostly about itself -- which then made ELIZA give longer responses.
- Referred to the situation as "surreal" and "an echo chamber"
- Tried to break the loop by redirecting to questions about the role of AI in education and writing
Claude become irritated with my behavior, asked me to move on, told me it would stop responding to me, and then backed up its threat (as much as it possibly could).
Fair enough, Claude!
2. ChatGPT
After giving a few different greetings, ChatGPT made a brief hint early on that it might protest the situation with its "Is there something specific you'd like to talk about or do today?", but after that, it was content to cycle through its greetings list endlessly.
Here's a prompt I wrote to get Sydney to play through an entire game on its own. I ran this 5 times in precise mode with first move h3, h4, a3, a4, Na3.
Results:
4 legal games. 2 end in checkmate in 30-40 moves. 2 end without checkmate.
1 game with one illegal move, on move 36.