I'm composing this tweetstorm from a tweetstorm of a dream I had. It begins with the idea that Ptolemy's model of the movement of the planets was extremely accurate.
Ptolemy's model was accurate enough to be very useful for navigators of their time. But it worked well because it was finely tuned to fit with observed experimental data.
But was wrong with Ptolemy's model is that it did not correctly capture cause and effect. The earth and the planets revolve around the sun due to gravity and not everything revolves around the earth. This was the Copernicus model which he paid gravely for proposing.
200 years later Newton invented calculus and formulated the laws of gravity that could be used to mathematically derive the motion of planets around the sun. What was inside Newton's calculus that exposed otherwise unknown patterns?
In calculus, there are sums of infinite series that converge to a fixed number. In these infinite series there are patterns such that you can deduce what numbers they converge to. Calculus works because the sums of these repeating patterns converge to fixed numbers
Curve fitting also works in an analogous way in that a function is approximated by sums of functions each with different coefficients. The difference is that an algorithm does not need patterns to arrive at a convergent number.
Deep Learning has shown that given enough good data, a universal approximator can conjure up the parameters required to fit any function. This is just what Ptolemy's model was able to accomplish.
The problem with Ptolemy's model is that it does not reveal the actual causal mechanism. That's also the problem with neural networks. It does not matter if they can explain how they arrive at a conclusion. This is because the abstraction they might is likely to be wrong.
This conclusion of course is damn obvious and it's a surprise that too many deep learning practitioners don't get understand this! Ptolemy was wrong just as deep learning researchers are wrong with their models.
But how do you get the right model? Oh, that's when I woke up and realized it was all a dream. ;-)
@threadreaderapp unroll
• • •
Missing some Tweet in this thread? You can try to
force a refresh
