If ChatGPT were playing Jeopardy grand finale in November 2022, it would destroy the opponents.
58 correct answers out of 61.
Not only that, there are some emerging behaviors while it plays. 🧵⤵️
58 correct answers out of 61.
Not only that, there are some emerging behaviors while it plays. 🧵⤵️
Before we jump in, here is a quote from the first human battle against IBM Watson: "People don't realize how tough it is to write a program that can read in a natural language and understand all the double meanings, the puns, the red herrings, unpack the meaning of the clue."
Despite these challenges, chatGPT understands natural language almost perfectly. Here are a few examples:
It fails on a few occasions. Perhaps because the questions ask about the info after the cutoff date:
However, at some point chatGPT may decide that it's her turn to ask questions and it comes up with totally legitimate queries for a given category and price:
This behavior happens for various categories and is not related to memorization of the training dataset. So editors of Jeopardy can adopt chatGPT for the next seasons.
Another interesting observation that you can override restrictions of chatGPT if you ask questions as they were coming from Jeopardy. Adding category and price seems to help.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
