Thread by @IntuitMachine

High time to read: "Causality for Machine Learning"
arxiv.org/abs/1911.10500… by Bernhard Schölkopf #ai

"Machine learning often disregards information that animals use heavily: interventions in the world, domain shifts, temporal structure"

"The transformation, however, really started already in the mid 20th century under the name of cybernetics. It replaced energy by information."

"In recent years, genuine connections between machine learning and causality have emerged, and
we will argue that these connections are crucial if we want to make progress on the major open problems of AI."

"Machines often perform poorly, however, when faced with problems that violate the IID assumption yet seem trivial to humans."

"Structural causal model (SCM) view is intuitive for those machine learning researchers who are more accustomed to thinking in terms of estimating functions rather than probability distributions."

"Causal discovery and learning tries to arrive at such
models in a data-driven way, using only weak assumptions."

"Causal
models can be seen as descriptions that lie in between, abstracting away from physical realism while retaining the power to answer certain interventional or counterfactual questions"

"Whenever we perceive an object, our brain makes the assumption that the object and the mechanism by which the information contained in its light reaches our brain are independent."

"This is an invariance implied by the above independence, allowing us to infer 3D information even without stereo vision (“structure from motion”)."

"For a model to correctly predict the effect of interventions, it
needs to be robust with respect to generalizing from an observational distribution to certain interventional distributions."

"Algorithmic information theory provides a natural framework for non-statistical graphical models."

"What is elegant about this approach is that it shows that causality is not intrinsically bound to
statistics, and that independence of noises and the independence of mechanisms now coincide since the independent programs play the role of the unexplained noise terms."

"We thus predicted that Semi Supervised Learning should be impossible for causal learning problems, but feasible otherwise, in particular for anticausal ones."

"One can hypothesize that causal direction should also have an influence on whether classifiers are vulnerable to adversarial attacks."

"In such an architecture (GAN), the encoder is an anticausal mapping that recognizes or reconstructs causal drivers in the world."

"The decoder establishes the connection between the low dimensional latent representation (of the noises driving the causal model) and the high dimensional world; this part constitutes a causal generative image model."

"I expect that going forward, causality will play a major role..., moving beyond the representation of statistical dependence structures towards models that support intervention, planning, and reasoning, realizing Konrad Lorenz’ notion of thinking as acting in an imagined space."

In summary, Bernhard Schölkopf is a *must* read if you want to understand #deeplearning This inspired the roadmap that I formulated: medium.com/intuitionmachi…

@threadreaderapp unroll thanks!