1/ It might be useful to look carefully at the logic of proving an "impossibility theorem" for NN, and why it differs fundamentally from Minsky/Papert perceptrons. The way we show that a task is impossible in Rung-1 is to present two different data-generating models (DGM) that
https://twitter.com/Plinz/status/1411811745077686272
2/ generate the SAME probability distribution (P) but assign two different answers to the research question Q. Thus, the limitation is not in a particular structure of the NN but in ANY method, however sophisticated, that gets its input from a distribution, lacking interventions.
3/ This is demonstrated so convincingly through Simpson's paradox ucla.in/2Jfl2VS,
even pictorially, w/o equations, in #Bookofwhy
Thus, it matters not if you call your method NN or "hierarchical nets" or "representation learning", passive observations => wrong answers
even pictorially, w/o equations, in #Bookofwhy
Thus, it matters not if you call your method NN or "hierarchical nets" or "representation learning", passive observations => wrong answers
4/ Plus, the interventions needed must be PHYSICAL not ANALYTICAL. You cannot simulate the former with the latter. Reinforcement Learning deploys physical interventions and can take you to Rung 2; but not to Rung 3. Please read Simpson and have fun playing the Simpson machine.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
