Logical inference about neural networks as models of brain activity and behavior
– a saga in 8 or 9 tweets –
by me, soon to be vastly improved by @o_guest /0
We often reason about what models represent, learn, or ‘know,’ based on performance on a task, or whether the model approximates human behavior/brain activity. Then sometimes we also try to infer what algorithm the human mind and brain instantiates based on this. /1
The arguments roughly go like this: “if the model approximates human behavior/brain data, then it does what people do” (read: it achieved this by doing what people do or having what people have). This appears to be modus ponens. But it isn’t! I’ll get back to that later. /2
Modus ponens: “if the model does what people do, then it will approximate human behavior”. That is good. But if we grant that, why not modus tollens? “if the model does what people do, then it will approximate human behavior; it does not, ergo it didn’t do what people do” /3
We rarely see modus tollens in the literature. Rather, the argument tends to go “the model did not get all the data humans get, that's why it behaved differently.” (solidarity to @GuillermoPuebl6 who dealt w/ this a lot; & see important work by @IrisVanRooij@MarkBlokpoel ) /4
But the same “not-the-same-data-as-humans-get” worry holds for all modus ponens cases. Really it holds for all modelling. 🧨So why is modus ponens accepted as valid, but not modus tollens? /5
Actually what we are doing is affirming the consequent: “if the model approximates human behavior, then it does what humans do” /6
But there are quite literally infinite ways it could produce the same performance… that have nothing to do with what humans do. Thus, I give you the fallacy of AFFIRMING THE CONSEQUENT, and not modus ponens /7
Modus ponens
If models do what humans do, then they will approximate human behavior.
Modus tollens
If models do what humans do, then they will approximate human behavior; but if they do not, then they do not do what humans do. /8
Affirming the consequent
If models approximate human behavior, then they do or have what people do or have.
AKA “modus don’tens,” or the epic “modus trollens” @o_guest /9
• • •
Missing some Tweet in this thread? You can try to
force a refresh
🧵What we love about ChatGPT is us, the human mind; AKA self-love is blind
The popularity of ChatGPT inter alia underscores the power of the human mind - but not because LLMs are in any way a model of it – buckle up! 0/N
Humans tend to interpret signals in the environment as meaningful (and we have a hard time with randomness and such); 1/N
we want to imbue signals with depth and structure based on surface statistics and behavior – because that is our niche, our ethological drive to make sense of the environment and act on it 2/N
web.stanford.edu/class/cs99r/re…
p 1333: “Artificial intelligence has the same relation to intelligence as artificial flowers have to flowers. From a distance they may appear much alike, but when closely examined they are quite different..." 3/n