I was recently on a panel with several other professors and we were asked to give some tips to graduate students in machine learning. It got me thinking about why professors are so bad at giving advice. So here are some reasons why you should not take advice from professors. 1. Distribution shift: the strategies that helped us establish ourselves decades ago may be totally obsolete. This is especially true in a field like machine learning, which has been transformed beyond recognition in the last 20 years.

Some thoughts after (finally) reading the Go-Explore paper (arxiv.org/abs/1901.10995). 👇 Clearly this is an important piece of work, and the empirical results speak for themselves. For a long time, exploration methods have either been smart or scalable, but not both.

Rich Sutton has a new blog post entitled “The Bitter Lesson” (incompleteideas.net/IncIdeas/Bitte…) that I strongly disagree with. In it, he argues that the history of AI teaches us that leveraging computation always eventually wins out over leveraging human knowledge. I find this a peculiar interpretation of history. It’s true that many efforts to incorporate human knowledge into AI have been discarded, and that more tend to be discarded as other resources (not just computation, but memory, energy, and data) become plentiful.

Next up on my summer reading list: GAN Q-Learning (arxiv.org/abs/1805.04874), a recent addition to the growing trend of distributional reinforcement learning. I find this trend intriguing and potentially quite exciting. Unfortunately, this trend has also generated a lot of confusion. I keep hearing people talk about distributional RL as if it could help us model the agent's uncertainty about what it's learning, a clearly desirable feature that is missing in many canonical algorithms.

So...I just read the "World Models" paper (arxiv.org/abs/1803.10122) from Ha & Schmidhuber. This is a nicely written, well researched paper with some cool/fun results. It also has a solid related work section and does a decent job putting the work into context. These are important strengths (and ones that are sadly not reliably present in a lot of recent deep learning papers). And yet...I don't like the approach proposed in this paper and feel pretty strongly that it is not the right way forward.