Thread by @IntuitMachine

Peter Godfrey-Smith in his book Other Minds speculates that consciousness exists in a continuum from single-cell animals all the way to complex multicellular creatures such as humans.

Hidden in the idea of a continuum of capability is an assumption of every agent has the same innate capability but is only restricted by degree.

When we think of artificial neural networks, there is what is known as the scaling hypothesis, which conjectures that general intelligence is achievable by even larger networks. arxiv.org/abs/2001.08361

This conjecture does have merit as evidence of the surprising 'zero-shot' (in scare quotes) capabilities of GPT-3. GPT-3 is qualitatively different from GPT-2 and it's a consequence of being a much bigger network. Quantity has a quality all its own.

We do know constitutes an artificial neural network, in contrast, we don't know as much of what constitutes the cell or the brain (a massive collection of neuron cells) of a living biological creature.

I admit though there is certainly an appeal of these continuum arguments. It is more persuasive than an argument that is absent this feature. An example of the kind without a continuum is Chomsky's Merge theory. That there's a new feature in humans that allows for language.

There are a lot of interesting continuums that relate to intelligence. There is the consciousness one that Godfrey-Smith mentions, there is also the agency continuum that is described by @drmichaellevin aeon.co/essays/how-to-…

There are many very counterintuitive things in biology. One is that every living thing constructs itself from a single cell. All the complexity of a multicellular creature is captured in a single cell.

If a single cell can express this kind of complexity, can it not also express a kind of cognition necessary for constructing a general intelligence? Rather than think of a cell as simple (which it is not), think instead that it's complex. Complexity begets complexity.

Complexity leading to complexity seems like a circular argument. Our reductionist bias would rather have simplicity begets complexity. Of course, this argument has been resisted also for centuries. So let's rephrase, complexity leads to other kinds of complexity.

Artificial neural networks has shown how one gets complexity from simplicity. Deep learning approaches have led to mastering Go and predicting protein folding. But are these problems in the same class of problems that living things solve?

Artificial neural networks do not begin their existence by solving the problems pertinent to life. Every synthetic neuron performs an exact calculation of a measure of similarity. Billons of these measures lead to curve fitting in high dimensions.

It's like a fluid filling the shape of a complex container. It's a kind of information processing that is perhaps very alien to how biological creatures process information. It is not of this world. Just as conventional Von Neumann computers were once not of this world.

But what do these deep learning networks not do? They don't create abstractions. They create models like Ptolemy did, but not like Copernicus. medium.com/intuitionmachi…

Deep Learning systems are inductive machines. Von Neumann computers are deductive machines. 100 years ago, C.S. Pierce enumerated a third kind of inference which he dubbed as 'abduction'.

What is an abduction machine? Are biological cells abduction machines? If they are abduction machines, how do they perform abduction?

Douglas Hofstadter's latest book 'Surfaces and Essences' is about how analogies are the core of cognition. Is what Hofstadter describes as analogy the same thing as abduction? Are cells also analogy making machines?

The complexity of any agent that predicts its environment is that it must construct for itself a descriptive model of the phenomena it needs to predict. The mechanistic way is to just gather universal approximations and curve fit.

The limitation of the approach is that it does not scale in capturing the combinatorial complexity of reality. Something that can perform analogies can take models developed in one context and apply them successfully in 'out of distribution' contexts.

This is what humans are able to do as well as many other animals. Is there thus a continuum, like we see in the inductive continuum in artificial neural networks, for something like analogy or abduction?

@threadreaderapp unroll