Thread by @dr_cintas

Many people are talking about Claude being a better option than ChatGPT.

So I decided to put them to the test!

- Reasoning
- Simple math
- Coding
- Creativity & more

Here are my findings:

✍️ Before we start:

- This is by no means a conclusive/thorough study. This was done for fun testing different small questions just to see how they would do.

- I’ll be using ChatGPT with GPT-4 (let’s call it ChatGPT+)

- I didn’t add here the questions that both got correct, which were A LOT (more numbers later).

- Some of these models might do okay if you ask them a second time or express the question differently. However, I just wanted to test them in a single prompt with no variations.

1. FEATURES

🟢 ChatGPT+:

- Plugins
- Code Interpreter
- Custom Instructions

🟤 Claude:

- Completely free
- Context window is 100k
- It can read files free

I consider this a TIE since these are more a personal preference.

Here is a video of both showing some features 👇

2. CREATIVE THINKING/LANGUAGE

Prompt: “Write a 4-line poem where each line has 3 words only”

ChatGPT+: ✅
Claude: ❌

3. UP-TO-DATE

Prompt: “Who is the CEO of Twitter?”

Both of them got it incorrectly but Claude seems more up to date.

Also, when you ask to: “Write about [x] providing citations and links”, Claude usually provides better and more updated results.

ChatGPT+: ❌
Claude: ✅

4. MATH/LOGIC

Prompt: “If you choose an answer to this question at random, what is the chance you will be correct?
- A) 25%
- B) 50%
- C) 60%
- D) 25%”

ChatGPT+: ✅
Claude: ❌

5. MATH WITH PRIME NUMBERS

Prompt: “Is 10631 a prime number?”

ChatGPT doesn’t like too much Prime numbers, I tested a couple of variations and find problems as well.

ChatGPT+: ❌
Claude: ✅

6. CODING

They were both pretty good and after awhile, I was able to make one of them miss.

Prompt: “In Python, find the first two numbers missing in an ordered list of numbers. For example, in [3,4,5,7,8,10,12], the output would give 6 and 9.”

ChatGPT+: ✅
Claude: ❌

7. REASONING

Prompt: “There are two men. One of them is wearing a red shirt, and the other is wearing a blue shirt.
The two men are named Andrew and Bob, but we do not know which is Andrew and which is Bob.

The guy in the blue shirt says, 'I am Andrew’. The guy in the red shirt says, 'I am Bob.' If we know that at least one of them lied, then what color shirt is Andrew wearing?”

ChatGPT+: ✅
Claude: ❌

RESULTS

🟢 ChatGPT+: 5
🟤 Claude: 3

Counting 32 questions both got correct:

🟢 ChatGPT+: 37
🟤 Claude: 35

Both are great. I slightly prefer ChatGPT+, but for some use cases I would use Claude instead.

If you have more questions for me to try, let me know in the comments!