Thread by @IntuitMachine

Science is being eaten up by deep learning. A fact that nobody can ignore.

But what's unfortunate is that nobody understands deep learning well enough to set up the experiments and interpretations correctly! It's damn good at making predictions but damn terrible at explaining anything!!

The uncertainty principle of deep learning is that the more generalized one's network, the least likely it's interpretable! medium.com/intuitionmachi…

That's because this is the nature of models that curve fit their data. Anyone who doesn't understand this has their head in the sand. medium.com/intuitionmachi…

Models that are interpretable are causal models. But causal models are descriptive models, not generative models. The origin of these models arises through abduction and not induction. medium.com/intuitionmachi…

The difference between a map and its territory is that a map changes when its territory changes, not the other way around. There are certain territories that are so complex that we have trouble creating a map.

But machines do not need maps. It's like Amazon warehouse bots that don't need orderly place items in categories. They have limitless memories and don't need the mental aids (i.e. categorization) that humans need.

There are two kinds of explanations, the intuitive kinds that give us the feeling of understanding. An example of this is the analogy that an atom is like a solar system. Where an electron 'orbits' around a nucleus.

Then there are explanations with reach. These are the kinds based on first principles and allow one to derive new consequences without ever requiring a real-world simulation. The conservation laws of physics are like this, they aren't intuitive but are useful explanations.

There are meta-patterns throughout nature that recur at different levels of complexity. Curve fitting algorithms are blind to these patterns. This is because these meta-patterns are baked into the architecture of the algorithm. It's not introspectable but a matter of habit.

The habits that are useful recur at different levels of complexity. Biology continues to reuse the same algorithms at multiple levels of complexity. The stuff that single-cells do to avoid predators is an example of an algorithm that's reinvented in multicellular creatures.

Algorithms are rediscovered all the time at different layers. The reason why higher levels can't just reuse lower-level algorithms is because they are composed of different stuff and interact differently.

The fallacy that most people have in their mind is that biological systems are composed of simpler stuff. So we are in constant surprise when the more we peer into single cells and neurons, the less we understand.

Physics had the same surprise decades ago. The more they probed into the smallest of particles, the more complex their mathematics became. Most don't understand quantum mechanics, but it gets more complex when you investigate the standard model, supersymmetry or string theory.

The same thing is happening in biology. The more they try to dissect the parts of a single cell, the more they realize how little we actually understand. The complexity is enormous for something that can recreate an entire organism.

Our reductionist bias assumes that stuff can be broken up into simpler parts. But this agenda has hit a brick wall in biology and neuroscience. The simple parts are even more complex than we had previously imagined.

So we have complex parts that interact with other complex parts leading to emergent behavior that is also complex. The assumption of simple parts leading to complexity addresses only a narrow kind of complexity.

Not only do we have complex parts leading to emergent complex behavior. We have these systems folding back into themselves and generating even greater complexity. To add insult to injury, there's no master designer! These systems design themselves in a manner absent a mind.