Read on Twitter
Large Language Models (LLMs) have emerged as the cornerstone of the current Generative AI revolution. The big problem with LLMs is that they are, well, large. Really, really, large.
1/7
1/7
They require an enormous amount of high-quality data to train and even more unfathomably large amount of computational power.
2/7
2/7
For a while now there has been a hope that fine-tuning a *smaller* language model on the output of some of these large ones would be a computationally efficient way of bringing the promises of GenAI to a wider audience.
3/7
3/7
However, in a recent in-depth research project, that promise has been seriously questioned. It seems that indeed for a genuinely general-purpose GenAI, the really large LLMs can’t be that easily replaced.
4/7
4/7
“Overall, we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs.
5/7
5/7
In turn, we argue that the highest leverage action for improving open-source models is to tackle the difficult challenge of developing better base LMs, rather than taking the shortcut of imitating proprietary systems.”
6/7
6/7
Read more in the following article:
“The False Promise of Imitating Proprietary LLMs”, arxiv.org/abs/2305.15717
#ArtificialIntelligence, #GenerativeAI #LargeLanguageModels #LLM #GenAI #AI
7/7
“The False Promise of Imitating Proprietary LLMs”, arxiv.org/abs/2305.15717
#ArtificialIntelligence, #GenerativeAI #LargeLanguageModels #LLM #GenAI #AI
7/7
• • •
Missing some Tweet in this thread? You can try to
force a refresh
