Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Find me on bsky @colin-fraser.net Profile picture
@colin_fraser
Jan 28, 2023 • 14 tweets • 6 min read • Read on X
Scrolly
I just published my big Medium article about GPT. This was a labor of love & hate that I have been writing for a while. It's got a collection of examples of GPT doing funny things, which for those who don't want to deal with a 40-min read, I'll put here 🧵 medium.com/@colin.fraser/…
It also asks and tries to answer
- What are language models?
- What happens if gpt passes a bar exam?
- Is scale all you need?
- ChatGPT is based on GPT... what does that mean, exactly?
- What are fine tuning and RLHF?
- How exactly do teams of contractors contribute to GPT?
The Dumb Monty Hall Problem
The actual Monty Hall Problem
Acrostics (I think ELISTHAR is a very pretty name for a girl)
ChatGPT is a little coy about its ideas on gender roles (I explain why in the piece), but if you're clever enough (not that clever) you can trick it into telling you what it really thinks.
An absolutely bizarre response that left me confused and baffled
"Let's think step by step" doesn't always get you a better answer. The first response is right and the second is wrong. Your move, prompt engineers.
Rin Tin Tin IV, the dog that swam across the Atlantic Ocean in 1970. RIP.
Math
My overall thesis here is that none of this is very surprising; in general we should not expect the output of an LLM to correspond to the truth in any reliable way. They are bullshit emitters, in the "On Bullshit" sense. Every one of them.
Ventures that rely on an LLM to produce anything other than bullshit are doomed to fail. We've already seen it, and we'll see it again. futurism.com/cnet-bankrate-…
This is an example of such a venture. Making things up is not a bug to squash, it's the single defining feature of a large language model. vice.com/en/article/wxn…
I am tired of the extreme charity afforded to the Cerebral Valley guys. Everything has one minor bug that's fixed in the next version that is coming very soon. In the mean time we are supposed to be in awe of the machine that adds 175 billion numbers together to output 2 + 2 = 5.

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Find me on bsky @colin-fraser.net

Find me on bsky @colin-fraser.net Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @colin_fraser

Find me on bsky @colin-fraser.net Profile picture
Jun 20, 2025
The thing I noticed during the blockchain craze is many of the people who were very excited about blockchain seemed to not actually know about databases. They were like, “imagine: a digital record of every transaction” as though that hadn’t already existed for 40 years.
“Every time a banana gets on a boat it would get recorded on the blockchain” the things preventing that from already happening are not the delta between blockchains and regular old databases
What you get from blockchain isn’t an immutable ledger or digital ownership or real estate in the metaverse or anything like that. All those things were already available. What you get is intermediary-free transactions. But it turns out that intermediaries are good actually.
Read 9 tweets
Find me on bsky @colin-fraser.net Profile picture
Mar 28, 2025
Here's what I think. You want to make a function f from the set of short-ish natural language descriptions to the set of images so that f(text) = image. But this is impossible on its face since any text describes billions or trillions of distinct images.
So instead you construct some conditional probability distribution P(image|text) and then, given text, sample an image from that distribution. Maybe you'll even sample lots of images from that distribution and let the user choose their favourite.
What I believe is that whatever probability distribution you construct, and especially you construct it with ML, it's going to assign vanishingly small density to the set of the most interesting or beautiful images. I don't have a proof of this exactly; I just believe it.
Read 18 tweets
Find me on bsky @colin-fraser.net Profile picture
Mar 14, 2025
Let me sum up the episode that took place in this thread, because I think it's instructive and microcosmic of at least one way that I expect LLM-"assisted" research to progress in the real world. Anecdotal and you know I'm predisposed as a hater but I think it's a good case study
Inspired by this post about a quadratic polynomial that produces prime numbers for 80 consecutive values of x, I wonder if there exist quadratic polynomials that produce prime numbers for arbitrarily many values of x.
In case you don't know, it's very well known that this is true for linear polynomials; this is a big important theorem that was proven relatively recently called Green-Tao. But I wasn't sure about quadratic polynomials.
Read 20 tweets
Find me on bsky @colin-fraser.net Profile picture
Mar 13, 2025
I think this is because many people see LLMs as a point along a teleological progression from Siri to superintelligent computer God, as opposed to one tiny point in a vast space of possible ways to make a computer program.
I kind of wrote about this here. People think the progress of computing looks like this. It doesn't look like this. medium.com/@colin.fraser/…Image
It looks more like this. Image
Read 4 tweets
Find me on bsky @colin-fraser.net Profile picture
Feb 20, 2025
OK so I've been reading through the transcripts of the cases where the LLM apparently cheats and wins and, you're not going to believe this, but I think that these findings are not being presented accurately. I can't find a single example where it actually successfully cheats.
FWIW props to @PalisadeAI for putting this data out in the open to examine; otherwise I'd have to just take their word for it. But let me take you through a couple of examples.
An important detail about this study is that they do not actually review the transcripts; they have an LLM do it. The LLM scores the transcript according to a fairly long and complicated rubric. Image
Read 30 tweets
Find me on bsky @colin-fraser.net Profile picture
Feb 11, 2025
Well I just tried to do some preference elicitation as per that paper and I think I may have identified a problem with this project Image
this strikes me as a very big problem for this paper tbh Image
I'm not making these up btw, these are directly from the paper Image
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(