Share this page!

Enter Twitter Thread URL to Unroll

Needs to be the full URL like: https://twitter.com/threadreaderapp/status/1644127596119195649

How to get URL link on Twitter App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Santiago Profile picture
@svpino
Jul 19 • 9 tweets • 3 min read Twitter logo Read on Twitter
GPT-4 is getting worse over time, not better.

Many people have reported noticing a significant degradation in the quality of the model responses, but so far, it was all anecdotal.

But now we know.

At least one study shows how the June version of GPT-4 is objectively worse than… https://t.co/whhELYY6M4twitter.com/i/web/status/1…
Accuracy comparison between the March version of GPT-4 with the June version on the problem of determining whether a number is prime. In March, GPT-4 solved 97.6% of problems accurately, while in June, it solved only 2.4% of the problems.
Here is the original paper:



And you can reproduce the results using this Google Colab:

https://t.co/gffe08ZGty https://t.co/9Tjqx6AFvjarxiv.org/pdf/2307.09009…
colab.research.google.com/github/lchen00…
Image
Something that I've had in my mind for a while now:

The honeymoon phase with Large Language Models is over. People moved away from cool demos to building actual applications.

Our tolerance for mistakes has changed drastically and it will continue getting lower.
Great to hear from @OfficialLoganK acknowledging that @OpenAI is listening:

Here is an interesting theory.

For a bit more context, the June version of GPT-4 supports "Function Calling," a new feature released by OpenAI where the model has the ability to suggest calling a function to solve a problem.

There are hundreds (maybe thousands already?) of replies from people saying they have noticed the degradation in quality.

Browse the comments, and you'll read about many situations where GPT-4 is not working as before.
Good analysis by @DrJimFan here, and my reply with the most important takeaway in my opinion:

It's not just about fixing the regression issues, but @OpenAI should implement a reliable versioning system.

I wrote a follow-up to this post here:

Vote in this poll:

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Santiago

Santiago Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @svpino

Santiago Profile picture
Jul 20
Yes, GPT-4 seems to be getting worse.

But now we have new information. And well, it's complicated.

Yesterday, I posted about a study showing that GPT-4 success rate deciding whether a number is prime went from 97.6% in March to 2.4% in June.

The report also showed how the… https://t.co/6jPFYPoWI0twitter.com/i/web/status/1…
Is GPT-4 getting worse?
Check the following post for more information about the reason we misinterpreted the original study:



@sayashk and @random_walker did an excellent job breaking down the original findings and ran the experiment that shows that GPT-4 was never good at… https://t.co/xzxNJOVWrHaisnakeoil.com/p/is-gpt-4-get…
twitter.com/i/web/status/1…
OpenAI is extending the lifespan of the March version of GPT-4.

This is good! Anyone relying on that version will have more time to upgrade to the newest June version.

They are doing this "because of developer feedback."

Read 4 tweets
Santiago Profile picture
Jul 15
Photography will never be the same.

In 10 minutes, you can turn your photo gallery into unlimited, amazing pictures. For free!

How much imagination do you have? Image
Follow these steps to generate your photos:

1. Find a few photos of you. The more, the merrier.
2. Go to and get an API KEY.
3. Run the code in the notebook below (Upload your photos first.)

Here is the code: https://t.co/LdmtSZlMJ5tryleap.ai
colab.research.google.com/drive/1v45UprB…
The code is dead simple:

1. It fine-tunes a model with your photos.
2. Waits for the process to finish.
3. Generates a photo of you following a prompt.

Run this, and you'll have as many photos of you as your imagination will let you.
Read 4 tweets
Santiago Profile picture
Jul 11
How do you think companies are training their Large Language Models? Where do you think the data come from?

Web scraping.

This is one of the most valuable skills you can learn.

Here is how it works and how you can learn it for free:
The Web Data Masterclass is a collection of videos about web data and how to collect it:



You'll find tutorials and how-tos from leading data scientists and engineers like @MariyaSha888, @ykilcher, @Jeffdelaney23, and @kunalstwt.

And it's free! https://t.co/IAg4MdHIsZbrdta.com/web-data-maste…
A short introduction:

Web scraping allows you to get data from websites at scale.

This data is unstructured.

You use web scraping to collect and store it in a structured form like CSV or JSON.
Read 8 tweets
Santiago Profile picture
Jul 9
This is the unfortunate state of AI shitfluencing.

People with nothing to add to the conversation and zero originality pump content like this to farm followers.

I'm sad for everyone who believes them.
I normally leave these people, but I’m not going to put up with lies and exaggerations that prey on people who don’t know better.

I’ve had more than a few conversations with students that want to quit or never learn programming because they read that AI killed the practice.… twitter.com/i/web/status/1…
Many in the replies telling their personal anecdotes about sons and daughters that don’t want to learn because what they’ve read online.

It pisses me off. These scammers are doing serious damage.

If you know somebody that thinks AI will kill their programming job, show them… https://t.co/dDuKv3eiTBtwitter.com/i/web/status/1…
Read 4 tweets
Santiago Profile picture
Jul 7
You can now fine-tune an LLM without writing a single line of code!

A breakthrough in the open-source LLM space that can increase the speed of AI development and adoption by an order of magnitude.

Let me start from the beginning:

A Large Language Model comes out of the factory… https://t.co/DwBVtReKL0twitter.com/i/web/status/1…
Here is a link to an article with a step-by-step demo of fine-tuning a model without writing any code: .

Thanks to @monsterapis for partnering with me on this thread.blog.monsterapi.ai/no-code-fine-t…
80% discount on using the code: SANTIAGO at the time of purchase.
Read 4 tweets
Santiago Profile picture
Jul 5
Another deep learning breakthrough:

Deep TDA, a new algorithm using self-supervised learning, overcomes the limitations of traditional dimensionality reduction algorithms.

t-SNE and UMAP have long been the favorites. Deep TDA might change that forever.

Here are the details:
Dimensionality reduction algorithms like t-SNE and UMAP have been around for a long time and are essential to analyze complex data.

Specifically, t-SNE is one of the most popular algorithms I've seen used in the industry.

Hinton and van der Maaten developed it in 2008.
But now we can do even better!

Deep TDA is a technique created by @datarefiner.

It combines the power of self-supervised learning and Topological Data Analysis (TDA) to unlock new insights from complex datasets.
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

Send Email!

:(