r0zetta Profile picture
Mar 17, 2023 10 tweets 4 min read Read on X
Is GPT-4 intelligent enough to solve ARC (github.com/fchollet/ARC)- a collection of intelligence tests devised by @fchollet? I ran a few quick experiments to check.
#gpt4 #gpt #ai #ml
In this test, the solution is to output the portion of the grid containing a different coloured cell. Thus the answer to the right-hand side question would be a 6x6 blue grid containing a red cell at the second position in the fourth row down. Image
GPT-4 did provide a smaller grid containing the correct colours (1 and 2). However, it generated a 5x5 blue grid with one red cell in it that was obviously in the wrong place. Not correct, but the gist of its answer was along the right tracks. Image
To solve this next task, copy the dot colours horizontally from both left to right and right to left and place a grey pixel at the mid point. A pretty easy puzzle for humans to solve. Image
GPT-4's solution did use the correct grid size. It did fill in the correct rows. However, it didn't choose the correct sequence of colours. The correct solution would be:

00000000000
44444588888
00000000000
00000000000
66666599999 Image
In the third and final test I tried, the solution is to find the background colour of the area of the picture that contains the most off-coloured cells. The solution is always a single number. In this case, one that represents red. Image
GPT-4 came up with its own theory that makes no sense when you read it. Image
I provided tasks to GPT-4 in a textual manner as illustrated here. Each colour was represented by a number. Input and output examples were labelled as such. The final output was left empty and I ran inference from there. Image
We've seen examples of GPT-4 performing rather amazing understanding tasks against images. Thus, it will be interesting to retry these tests against the version of the model that accepts images as input in order to determine whether it does better.
Bottom line: we not ded yet.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with r0zetta

r0zetta Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @r0zetta

Oct 29, 2019
Interesting. I might have found a way to identify prolific troll accounts that use the same, or very similar replies over and over. This doesn't necessarily imply automation, but it can't be ruled out.
Here's another example. This time the tweets are not identical, but they contain quite similar patterns.
And here it is in action, just moments ago. Account created a few months ago, with 16k tweets already published... So many of these accounts exist and fly under Twitter's radar.
Read 4 tweets
Oct 22, 2019
Here's another thread containing troll replies from anti-EU fake/sockpuppet accounts. Many of these troll accounts have retweeted and posted almost identical content, and are trolling many other pro-EU threads.

Twitter is absolutely plagued with this stuff.
PSA: if you see accounts with large numbers (multiple thousands) of followers roughly equal to numbers of accounts following them, these accounts belong to follow-back rings.
Follow-back rings are largely comprised of fake accounts and are echo chambers designed to amplify content via mass retweets.
Read 11 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(