There's a mad rush in Machine Learning circles to say 'X is enough' or 'X is all you need'. We have 'Rewards is enough', 'Attention is all you need', 'Diverse training is all you need', 'Size is all you need' (Bitter truth/Scaling hypothesis).
These logical jumps are ignoring all the messy details. I'm guilty myself of some ideas like 'Intuition machines is all you need' or 'Empathy is the path to general intelligence'. These singular ideas are pleasurable because they make what's complex appear simple.
Brains and Deep Learning are both live-wired systems and they are anything but simple. They are complex adaptive systems and require the entire kitchen sink of technology and mathematical models to get a handle on.
So let's not gloss over complexity by conjuring up short sound-bites of how live-wired systems are supposed to work.
BTW, add 'foundation models' to the original post! That is, we've identified foundational models but we actually don't know what they are. We know it only when we see it. Aka... Intution blanket.
Perhaps as humans with cognitive biases, we cannot avoid framing our research agenda as short bite-size digestible nuggets. This is the reality of any scientific endeavor. If you can't express your agenda the time it takes to ride an elevator, then you are at a disadvantage.
So the practice of science has similarities with the practice of philosophy. Those who can craft an appealing and intuitive narrative of complex systems are in a favorable position. He who can make people see the forest instead of the trees wins the recognition.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
