Thread by @Grady_Booch

Some thoughts about contemporary approaches to deep learning.

I celebrate the amazing advances we have seen in recent years with DL. CleRly, this demonstrates that connectionist models of computation have enduring value.

However...

Virtually all of the more spectacular applications of DL are for what I call signal AI: the classification of patterns within static images, video streams, and data streams (the latter being voice in particular).

From an architectural point of view, this simply means to me that DL offers efficient mechanisms for that class of problems...but DL as we practice it today does not necessarily transfer to other sorts of problems.

Most notably, decision planning and analytics are already well served by symbolic mechanisms of computation (mechanisms that also more easily address identifying rationale for those outputs).

I suppose what gives me the greatest pause is that DL greatly abuses the semantics of what a neural network is and what it is not. Albeit biologically inspired, the neural networks used by the vast majority of DL algorithms are but faint and distant shadows of biological neurons.

Most DL algorithms use binary valued neurons; biological neurons are spiking...and we have only begun to understand the information content of these far richer signals.

Most DL algorithms use this one class of neurons; in the human nervous system, there are dozens of different classes of neurons that operate in subtly different ways.

In conclusion, I celebrate the amazing advances in DL.

But we would be well advised to temper our celebration with the understanding that there is so much more that we do not yet know.

And for me, that is an exciting call to action to undertake that journey of understanding.