Have you ever realized that the big ideas in Deep Learning are just formulations of very old ideas. Generative Adversarial Networks (GANs) @goodfellow_ian is just Hegel's dialectic (i.e. thesis and antithesis hence synthesis).
That @DeepMind @demishassabis Alpha* self-play is just the Socratic method. en.wikipedia.org/wiki/Socratic_…
That skip connections are just a formulation of small-world networks.
That random initialization is just random projections (see: en.wikipedia.org/wiki/Locality-… ).
That Transformers are just C.S.Peirce indexical signs.
That the latent representation of neural networks are just eigendecompositions.
That Q-learning is just the Hamiltonian Jacobi Equations.
That regularizations are just constraints in the method of Lagrange?
That word2vec and language models are just Ludwig Wittgenstein's observation about words and the contexts that they are used.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
