Thread by @NeuroNimer

Now that #ChatGPT4 is out, my timeline is full of hyperventilating hyperbole about it "replacing doctors", "writing papers", "acing exams", making [x profession] obsolete," etc.

They couldn't be more wrong.

To understand why, buckle up for a (far too) long thread.🧵

I'll explain the architecture of LLMs & their limitations and give you solid examples of why you - or anyone - have nothing to fear from #ChatGPT & explain my motivation in writing this (besides my sheer irritation at the ignorance of the claims clogging my- and your - timeline).

#ChatGPT4 is an excellent tool still very much in its infancy. Like many other AI & AI-adjacent tools, it is touted to perform miraculous tasks that it was _not_ meant to perform and has no capability of performing. Claiming otherwise is ignorance at best and grift at worst.

Let's start with the claim that ChatGPT can "replace doctors." Here is a question about the use of a drug in a certain pathology.

Not only did it mischaracterise the pathology, but it gave us the wrong treatment recommendation. And mischaracterised the study it based its recommendation on.

This is not a one-off. Let us try it with another typical exam question every junior neurosurgeon should know.

And another. Even if you have ZERO domain knowledge of the subject matter discussed, if you scratch the surface...

... you'll find that it is making it up. The second landmark study by @pja_hutch @ag_kolias et al definitely exists, is widely cited, and changed practice. The other two? Not so much. They don't even exist. It made them up. Out of thin air. Yikes.

I could give you literally hundreds of examples from a multitude of domains, including code writing, where it is supposed to excel. So doctors, programmers, essayists, etc, your jobs are safe. ChatGPT will not be passing the neurosurgery FRCS any time soon. In fact...

it wont be writing any meaningful scientific literature either. But why does it seem to be so fantastic at times, while giving us absolutely rubbish answers at others? The answer lies in their architecture.

What is ChatGPT exactly? It is a large language model (LLM) developed by OpenAI, based on the GPT (Generative Pre-trained Transformer) architecture. Transformers are a deep learning architecture initially specifically designed for natural language processing (NLP) tasks.

The first transformer paper "Attention Is All You Need" is (rightly) treated with a cult-like reverence amongst AI researchers. It introduced a new architecture based on the concept of self-attention. Previous NLP models, such as recurrent neural networks (RNNs) and ...

... convolutional neural networks (CNNs), had limitations in their ability to capture long-range dependencies between words in a sentence or paragraph. The key innovation of the transformer architecture is the self-attention mechanism, which allows the model to "attend" to...

different parts of the input text and weigh the importance of each token (word) in the context of the entire text. The attention mechanism allows the model therefore to identify the relationships between words and sentences, and use this information to make predictions ...

about the next word or phrase in the text when generating new text.

Next token prediction is a fundamental task in transformer-based LLMs. The model is trained on large amounts of text data (GPT 3 has 175 billion(!!!) parameters), and learns to predict...

the most likely next token (i.e. word or phrase) given the current context. This involves a complex series of computations and weight adjustments, based on the relationships between tokens in the input text.

Transformers can analyse entire text passages or documents as a whole, making them more effective than earlier NLP models that processed text in a linear manner. This gives them the ability to accurately identify the most likely next word or phrase, and generate text that...

follows the overall structure and style of the input data. This is why you can, for example, generate a discharge letter in the style of Snoop Dogg

...or an op note in the style of Rihanna.

That's basically what it all boils down to. Next token prediction. I am simplifying it massively, because the architecture is immensely complex, the mathematics mind-bending, the code just *chef's kiss*.

Transformers are a work of art. A thing of absolute beauty. But magic they are not. They very much depend on the accuracy and size of their input training data.

So please ignore the hyperbole, the tech bros proclaiming the death of entire industries, the grifters chasing clout on the back of the hardworking people who conceived ChatGPT and other LLMs, and the frankly embarrassing takes regarding GPT and medicine/science.

It's important to note that openAI itself makes no such claims but (unlike AI tech bros) commendably and specifically speaks about their own model's limitations.

So why write this thread? Why not let the grifters grift and the ignorant shout into the void? Because we've seen this before. And it did not end well.

historyofdatascience.com/ai-winter-the-…

Media hype around AI led to previous AI winters. They overpromised and our predecessors underdelivered. It was genuinely not their fault. The giants of AI on whose shoulders we stand were usually cautious and scientifically sound individuals. The media hype around their work...

killed AI research for years. It would take a herculean effort of countless tireless individuals to bring back AI research back on track and to where we are today. And where would we be without them? How much further would we be today had the #AIwinter not happened?

Would you take a prescription for medication from someone who didn't study pharmacology, biochemistry, physiology, etc.? No? So why would you take AI advice from someone who didn't study it in depth? Always, always check your sources.

#AI will have the capability to change every facet of our life. Let us safeguard AI research by not overpromising and underdelivering. Let's fight the hype by being scientific about our approach to AI. Our future - and the future of sound AI research - depends on it.