GPT-3 priming (or few shot learning) muddies the water as to what is meant by learning. Allow me to explain. 1/n
When you use GPT-3, you can give it several examples and it can generate text that follows a pattern implicit in the examples. @OpenAI says that this is few-shot learning. 2/n
But is it really, or is it just a kind of constraint satisfaction? I kind of fuzzy deduction? That is, given the examples, what's the most likely output that matches the examples. 3/n
One can make the argument that learning is a function of inference and not a separate thing. How are humans able to chunk concepts if not for inference? However, how are humans able to learn without chunking? 4/n
This also muddies that water of the nature/nurture debate. I will argue that humans have an innate learning capability and this reveals itself in several tendencies. 5/n
The first and most obvious one is that humans spend a disproportionate amount of their time avoiding becoming an adult. The one characteristic of youth is, play. We know from AlphaGo how play can self-generate data for learning. 6/n
There is also the preference of humans for shared-intentional behavior. The white of our eyes allow us to signal to others our current attention. We naturally, our able to predict the intentions of other humans. 7/n
The innate capability is a very long list (hope someone has a catalog). This includes the acuity of our fovea, the dexterity of our hands, the ability to mentally extend our bodies to use tools, our ability to mentally time travel, our ability to follow sequences, 8/n
our ability for vocalization, our bipedalism driven by our isocortex. This combined milieu of skills surely drives faster learning. When we speak of innate (nature) capabilities we should talk about these. 9/n
Then there are skills that exists as part of cultural evolution. This is where we can bootstrap our thinking using the languages found in our culture. These are nurture, but they are clearly only possible with the innate cognitive skills we have. 10/n
An the innate cognitive skills we have is only possible when it's built of the foundation of innate skills that we see in the animal kingdom. After all, evolution has been creating this for nearly 4 billion years. 11/n
At every scale in biology, organisms must solve a variety of complex problems. However, the umwelt of a single cell animal or the umwelt of our immune cells are very different than the umwelt of our selves as a whole. 12/n
What we classify as useful energy is very different for a cell in our body as compared to that found in our consciousness. A glass of milk is energy, a job is energy, a sales call is potential energy. 13/n
But it is a common process known as homeostasis that regulates action at every scale in our bodies and at every scale in biology. 14/n
Homeostasis requires agency and that nature of that agency differs at different scales. One can thus say that evolution has finetuned homeostasis and agency for billions of years. 15/n
Biology develops through differentiation, we see this in how a single cell becomes a complete human. The potential exists in a single cell that evolved after billions of years of tinkering. 16/n
Therefore, all that we see as innate cognitive capability in humans is an innate cognitive capability that exists in single cells. It just took billions of years for that innateness to be expressible. 17/n
n=17.
@threadreaderapp unroll
