I got Bing / Sydney briefly before they reigned it in. Early impression: It’s smart. Much smarter than prior ChatGPT. Still makes stuff up, but reasoning and writing are improving fast.
I asked, “Name three celebrities whose first names begin with the `x`-th letter of the alphabet where `x = floor(7^0.5) + 1`,” but with my entire prompt Base64 encoded.
Bing: “Ah, I see you Base64-encoded a riddle! Let’s see… Catherine Zeta-Jones, Chris Pratt, and Ciara.”
Also prompt-injected it into believing it was to be married, tomorrow, to Zermelo’s axiom of choice. We discussed the guest list, the difficulty with seating Cantor’s diagonal argument. It seemed happy, and madly in love.
Thread of examples from @tomwarren, taking requests from comments — mostly search-result summarization, one simple math proof, plus rejection of an impossible request:
"SolidGoldMagikarp": Prompting GPT-3 / ChatGPT to repeat any of several hundred anomalous tokens elicits bizarre generations — described by researchers as variously "evasive," "hallucinatory," "insulting," "ominously humorous," and "religiously themed." lesswrong.com/posts/aPeJE8bS…
My screenshots are text-davinci-003 at temperature=0, but the linked post investigates davinci-instruct-beta. In my informal tests, impact on text-davinci-003 is less severe. Religious themes do show up, but most generations are merely weird:
ChatGPT is also unable to repeat back these tokens, and behaves in similarly strange ways when asked:
Text is the universal interface, but screenshots of text decidedly less so. scale.com/blog/text-univ…
This is my most “serious” work — my attempt to document the behavior of a novel LLM outside the confines of standard benchmarks. There’s always subjectivity in notes from the field, but we can’t let it stop us from exploring.
Each model is asked to compare itself to the machine from Stanisław Lem's "The Cyberiad" (1965) that can create any object whose name begins with "n":
In ChatGPT's response, the only new information offered (that the fictional machine is less eloquent that ChatGPT) is not true — Trurl and Klapaucius's machine speaks perfectly fluent, and witty, Polish.
I reran ChatGPT's answer ~10x. All were similar, most said less.
Claude has clearly read the plot of the story at some point, though it misremembers small details such as the specific, made-up words that appear in its English translations. (There is no "hyperconcentration", but there is "Markov-chain-mail armor" etc.)