Gary Marcus (@GaryMarcus) frames his points using the same vocabulary as computation. As a consequence, it appears overly general, covers everything and thus says nothing. But let me attempt to use a different vocabulary.
@GaryMarcus Let me use the words naming, identity, equality and methods instead of his vocabulary of variable, instances, binding and operation. Any OOP programmer will instantly recognize the four and see why it is useful.
@GaryMarcus In the above slide, @GaryMarcus argues that human cognition must be capable of the four operations. Few will disagree with this assertion. Few will also disagree with the assertion that the brain is computational (if we use Marcus' vocabulary).
@GaryMarcus But the GOFAI top-down argument will claims that the computational components that achieve these capabilities must be designed into the architecture. The connectionist by contrast argue that these capabilities must be evolved into the architecture.
@GaryMarcus So it seems like GOFAI is analogous to a neo-creationist argument. This is the primary philosophical point that DL researcher disagree with. You must evolve functionality and not bake it into the architecture.
@GaryMarcus The argument on the side (linguists included) is that we've amassed knowledge about this world. Why shouldn't we use it to design our cognitive systems? These are the two point of views that straddles the extremes of how technology arises.
@GaryMarcus The hybridization argument (which I call dual process) is a pragmatic approach that that says that one can do work in between the extremes. Nobody will disagree with this. But people will disagree with the underlying philosophy. That is what we see in the #AIDebate
@GaryMarcus No let's dig deeper. Does DL support naming, identity, equality and methods that @GaryMarcus argues is essential to human cognition. Equality and methods are the most obvious that DL supports. DL can discovery the equality of instances and the methods available for interaction.
@GaryMarcus Alpha* is an existential proof that DL can determine the value (i.e. equality) in a board position and the action (i.e. policy) required for interaction with the board.
@GaryMarcus What about identity? Does DL recognize the identity of individuals or the identity of sets of individuals? We have to be careful here because identity is not the same as equality. But if DL can identify faces, then should it not be able to capture the notion of identity?
@GaryMarcus Finally, the most troublesome of the four is naming. Naming is a social construct that is required for coordination behavior. It's not humans who understand names, dogs can grok this too. A sheepdog knows the names of the sheep. Bengio alluded to this and called it 'references'.
@GaryMarcus To conclude, Marcus' naming,identity,equality and methods are all covered by existing research. Perhaps not completely covered at the present but at least partially. IMHO, I don't think the line in the sand that Marcus has drawn can't be toppled in the near future.
@GaryMarcus Unless @GaryMarcus redraws the line in the sand.
@GaryMarcus Let's explore further. Can "liquid brains" solve these four? Examples of liquid brains are bacteria, ants and the immune system. Perhaps naming would be problematic on the other hand, how do liquid brains coordinate if not for a mechanism that would be analogous to naming?
@GaryMarcus The Pi-calculus which is a computational model of distributed processes is built on a concept of a 'vocative name'. It is this concept that is essential for coordinated computation. In fact, biology is distributed processing.
@GaryMarcus Also covered briefly in the #AIDebate is this notion of DNA encoding. How is DNA able to encode instructions to reconstruct the entire body and the entire brain? One might argue that there is insufficient information in DNA.
@GaryMarcus But that's like arguing that there is insufficient information in the primordial universe to evolve minds that can contemplate about the universe. Clearly there's is scarcity in understanding of what information can contain.
@GaryMarcus The nature of emergence is not well understood. Emergent capabilities arise because evolution increases the likelihood of useful combinations arising from previously invented useful combinations.
@GaryMarcus A process analogous to how Deep Learning makes use of hierarchy (i.e. the reuse of computations from lower layers) is how evolution builds up from existing biology.
@GaryMarcus Evolution creates generative models that allows living things to reconstruct themselves from the information in the DNA. This reconstruction process does not invent new information but rather interprets existing information in a multitude of ways.
@GaryMarcus Information in a cell isn't lost. The differentiated cell (a heart cell or a brain cell) just reinterprets the information that exists in the DNA. A body constructs itself from information within. Unlike human technology that is constructed from information from outside.
@GaryMarcus Generating itself from within seems magical. To make it seem less magical, think instead of recombining existing algorithms in different ways. The generative instructions are all in the cell, it's just a matter of evolution to discover which expressions are useful.
@GaryMarcus The mechanisms of human cognition were invented in the Cambrian explosion. It just took 400 million years to have it express itself in homo sapiens.
@GaryMarcus The reason Moravec's paradox exists in that our science sees only a shallow interpretation of what it took biology 4 billion years to create. The failure of GOFAI is that it adopted an anthropocentric model of cognition.
@GaryMarcus @threadreaderapp unroll please
