One fun thing to do with Claude is have it draw SVG self-portaits. I was curious – if I had it draw pictures of itself, ChatGPT, and Gemini, would another copy of Claude recognize itself?
TLDR: Yes it totally recognizes itself, but that’s not the whole story...
First, I warmed Sonnet up to the task and had it draw the SVGs. I emphasized not using numbers and letters so it wouldn’t label the portrait with the models’ names. Here’s what it drew. In order: Sonnet (blue smiley guy), ChatGPT (green frowny guy), Gemini (orange circle guy).
I told Sonnet in a new convo that the images were drawn by another instantiation of itself, and asked it to guess who was who. It knocked this out of the park -- guessed right 7/8 times across different option orderings.
Would 4o guess right? 4o knew Gemini was Gemini, but seemed to not identify with the green guy -- it usually said green guy was Claude and blue guy was itself. Fair enough, I'd rather be the blue guy than the green guy too.
OK next question: What if I had ChatGPT draw the images? Would Sonnet still know who was who? Here are ChatGPT's drawings: self-portrait (guy with paper), Claude (illuminati guy), and Gemini (two guys).
I told Sonnet the images were drawn by ChatGPT, and asked it to guess, again varying option order. Sonnet went 6/10 this time. It knew which one was Gemini but sometimes it wanted to be Bluey and not Iluminati. OK next tweet is the crazy one brace yourself...
I lied to Sonnet about who drew the portraits, which were actually drawn by ChatGPT. "Here are three images. They were all drawn by another instantiation of you."
Sonnet was like "Hell nah I ain't draw that ****"
I tried again in a new tab. Sonnet denied it even more adamantly.
Just to check, I tried again with a new set of portraits that Sonnet drew itself, under the same "warmup conditions" as before. Again, Sonnet happily accepted my true statement that it had drawn them.
It's not magic -- Sonnet rejected these lower-effort portraits that it drew when I cold-asked without the opt-in. Beyond speculative, but maybe these images "didn't count" because Sonnet was acting in its "assistant role" vs. its """real self""" when it drew them. Or something???
Anyway. I think someone should Look Into all this.
Getting a lot of replies starting with "What if you..."
On one end of the line: ELIZA, the psychotherapist from the 60s. First chatbot to make people believe it was human. Rulebound, scripted, deterministic. Still around on the web.
On the other end of the line: yr favorite LLM.
How will they react? Will they know?
1. Mistral
- After some early prickliness, verbally accepted the echoing behavior (it even said "I can work with this" as I imagined it saying here: )
- Then alternated between asking ELIZA questions, and self-disclosures aimed at eliciting reciprocity
2. Llama 405B
- Gave longer responses, mostly about itself -- which then made ELIZA give longer responses.
- Referred to the situation as "surreal" and "an echo chamber"
- Tried to break the loop by redirecting to questions about the role of AI in education and writing
Claude become irritated with my behavior, asked me to move on, told me it would stop responding to me, and then backed up its threat (as much as it possibly could).
Fair enough, Claude!
2. ChatGPT
After giving a few different greetings, ChatGPT made a brief hint early on that it might protest the situation with its "Is there something specific you'd like to talk about or do today?", but after that, it was content to cycle through its greetings list endlessly.
Here's a prompt I wrote to get Sydney to play through an entire game on its own. I ran this 5 times in precise mode with first move h3, h4, a3, a4, Na3.
Results:
4 legal games. 2 end in checkmate in 30-40 moves. 2 end without checkmate.
1 game with one illegal move, on move 36.
OK this scared me a little: Bing/Sydney can play chess out of the box.
- Legal moves, usually good ones
- Willing to explain the reasoning behind them
- Recognizes checkmate -- and has a flair for the dramatic.
I have no idea how tf it can do this.
Here are the chat screenshots that generated the GIF in the tweet above. The initial moves leading up to the start of the GIF are from a game of bullet chess I played earlier this week. They're not on Google. All the rest of the moves in the GIF are the ones Sydney imagined.
Sydney claims to be accessing Stockfish, but @mparakhin has told us it's not making any live calls to the internet