steven t. piantadosi Profile picture
Jun 14, 2022 22 tweets 7 min read Read on X
Everyone seems to think it's absurd that large language models (or something similar) could show anything like human intelligence and meaning. But it doesn’t seem so crazy to me. Here's a dissenting 🧵 from cognitive science.
The news, to start, is that this week software engineer @cajundiscordian was placed on leave for violating Google's confidentiality policies, after publicly claiming that a language model was "sentient"…
Lemoine has clarified that his claim about the model’s sentience was based on “religious beliefs.” Still, his conversation with the model is really worth reading:…
The response from the field has been pretty direct -- "Nonsense on Stilts" says @GaryMarcus
Gary's short piece cuts to the core of the issues. The most important is over-eagerness to attribute intelligence. This experiment from the 1940s illustrates it: people perceive beliefs, emotions, intentions, even when shown only moving shapes.
But, beyond that warning, I'm not sure I agree with much. First, it's just not true that systems which only do "pattern matching" are necessarily cognitively impoverished. Image
In fact, we've known since the earliest days of computing that just pattern matching (e.g. systems of rules that match patterns and rewrite) is capable of *arbitrary* computation.…
So even a model that learns really well over "pattern matching" rules is potentially learning over the space of all computations.

(And that, btw, is a pretty good guess for what human learners do… )
This means that a smart "pattern matching" model might, in principle, acquire any computational structure seen in the history of cognitive science and neuroscience. Image
In other words, what matters is NOT whether the system uses “pattern matching” or is a “spreadsheet.” What matters is what computations it can actually learn and encode. And that’s far from obvious for these language models, which carry high-dimensional state forward across time.
Many have also doubted that large language models can acquire real meaning. The view is probably clearest in this fantastic paper by @emilymbender and @alkoller…
Bender and Koller use "meaning" to be a linkage between language and anything else (typically stuff in the world). Their "octopus test" shows how knowing patterns in language won't necessarily let you generalize to the world. ImageImage
I guess I lean agnostic on "meaning" because there ARE cognitive theories of meaning that seem accessible to large language models--and they happen to be some of the most compelling ones.
One is that meaning is determined, at least in part, by the relationships between concepts as well as the role they play in a bigger conceptual theory.
To use Ned Block's example, "f=ma" in physics isn't really a definition of force, nor is it a definition of mass, or acceleration. It sorta defines all three. You can’t understand any one of them without the others. Image
The internal states of large language models might approximate meanings in this way. In fact, their success in semantic tasks suggests that they probably do -- and if so, what they have might be pretty similar to people (minus physical grounding).
To be sure (as in the octopus example) conceptual roles don't capture *everything* we know. But they do capture *something*. And there are even examples of abstract concepts (e.g. "prime numbers") where that something is almost everything.
It’s also hard to imagine how large language models could generate language or encode semantic information without at least some pieces of conceptual role (maybe real meaning!) being there. All learned from language.…
Conceptual roles are probably what allows us, ourselves, to talk about family members we've never met (or atoms or dinosaurs or multiverses). Even if we know about them just from hearing other people talk. Image
For the big claim.... Not a popular view, but there's some case that consciousness is not that interesting of a property for a system to possess.
Some model happens to have representations of its own representations, and representations of its representations of representations (some kind of fancy fixed point combinator?)....
... and so what!

Why care if it does, why care if it doesn't.

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with steven t. piantadosi

steven t. piantadosi Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @spiantado

Feb 7
It is an amazing time to work in the cognitive science of language. Here are a few remarkable recent results, many of which highlight ways in which the critiques of LLMs (especially from generative linguistics!) have totally fallen to pieces.
One claim was that LLMs can't be right because they learn "impossible languages." This was never really justified, and now @JulieKallini and collaborators show its probably not true:
One claim was that they LLMs can't be on the right track because they "require" large data sets. Progress has been remarkable on learning with developmentally-plausible data sets. Amazing comparisons spearheaded by @a_stadt and colleagues:
Read 13 tweets
Jan 31, 2023
Really great job here, @GlassHealthHQ. Your technology is ready for clinical use. ImageImage
@GlassHealthHQ sounds like sleep apnea, says @GlassHealthHQ Image
Read 13 tweets
Dec 4, 2022
Yes, ChatGPT is amazing and impressive. No, @OpenAI has not come close to addressing the problem of bias. Filters appear to be bypassed with simple tricks, and superficially masked.

And what is lurking inside is egregious.

@Abebab @sama
tw racism, sexism. Image
It's not a fluke
Read 9 tweets
Dec 3, 2022
still just unbelievable
Read 7 tweets
Aug 26, 2022
Yeah, yeah, quantum mechanics and relativity are counterintuitive because we didn’t evolve to deal with stuff on those scales.

But more ordinary things like numbers, geometry, and procedures are also baffling. Here’s a little 🧵 on weird truths in math.
My favorite example – the Banach-Tarski paradox – shows how you can cut a sphere into a few pieces (well, sets) and then re-assemble the pieces into TWO IDENTICAL copies of the sphere you started with. Image
It sounds so implausible, people often think they've misunderstood. But it's true -- chop into a few "pieces" and reassemble to two *identical* (equal size, equal shape) spheres to what you started with.
Read 39 tweets
Jan 25, 2022
I am sooooooo excited for this paper. We've spent years developing a super fast program induction library. We use it to learn key pieces of language structure.

So much of what Chomskyan linguists say about learnability is totally wrong.

We show that a program-learning model can construct grammars and other computational devices from just observing utterances (sentences). It takes just a tiny amount of data to learn key patterns and structures from natural language which have been argued to be innate/unlearnable
We also show that this kind of learning model can acquire the patterns used in artificial language learning experiments, like @jenny_saffran, Aslin, and Newport.
Read 30 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!