I'm going to try to live tweet Matt Botvinick's (DeepMind) keynote talk at #CogSci2018. Wish me luck:
The title of the talk is "Nature and Nurture in AI". Botvinick starts by surveying the developments in AI that excited him and made him join DeepMind.
The first example is the success of convolutional neural networks in image recognition - a long-exist toolkit that was first applied successfully (with a few important tricks) in 2012.
Botvinick: recurrent neural networks have led to significant advances in machine translation - another existing connectionist idea that was recently shown to be scalable to rich real-world data sets. #CogSci2018
Botvinick: But what attracted me to DeepMind was the work on deep reinforcement learning. The first demonstration was neural networks that learned to play Atari games at superhuman levels, just from the pixels. #CogSci2018
Botvinick: The system was based on a neural network. Instead of getting labels after each trial, the system has to learn through rewards (did I get a point?) and exploration - reinforcement learning rather than supervised learning. #CogSci2018
Botvinick: The combination of reinforcement learning and deep learning was a genuinely new development, and later led to successes in Go and StarCraft. #CogSci2018
Botvinick: I was excited not just because of the inherent coolness of this research, but also because of the connections to psychology and neuroscience. Neural networks are about representation learning that supports generalization and inference, like in psychology. #CogSci2018
Botvinick: Neural networks acquire representations that are similar to biological representations (e.g., in the ventral stream of the visual system, Dan Yamins's work). #CogSci2018
Botvinick: The second piece of the puzzle is reinforcement learning, which again corresponds to biological systems (e.g., the dopamine system). #CogSci2018
Botvinick: After I joined DeepMind, I wrote a position paper titled "Neuroscience-Inspired Artificial Intelligence". #CogSci2018
Botvinick: I want to organize the points I want to make under the headings of nature and nurture. Nurture: abilities gained from learning and feedback from the environment; nature: all the rest, things that don't depend on the data you get. #CogSci2018
Botvinick: In AI, these are different strategies for building systems: nature builds components by hand into the system; nurture focuses on learning from data. Back and forth between these strategies in the history of AI. #CogSci2018
Botvinick: The project of knowledge engineering - codifying all of neural knowledge by hand, including intuitive common sense knowledge - turned out to be extremely difficult. #CogSci2018
Botvinick: The comeback of AI is driven by a radical turn in the other direction: let's learn entirely from huge amount of data (taking the nurture route). #CogSci2018
Botvinick: We have new ways of using data now. We don't need to manually label all of the examples: in reinforcement learning all you need in an environment that gives feedback, the agent generates its own data through exploration (that's the secret behind AlphaGo). #CogSci2018
Botvinick: AlphaGo plays against itself and learns from that experience: no need to generate an expensive manually labeled dataset. #CogSci2018
Botvinick: Self-supervision: making predictions about the environment and learning from the success or failure of those predictions. The system gradually constructs representations that are useful for prediction. #CogSci2018
Botvinick: (Example: Banino et al Nature paper on a reinforcement system that evolves representation that are similar to grid cells in entorhinal cortex.) #CogSci2018
Botvinick: We showed a system what a scene looks like from one point of view and had it guess what it would look like from another view. The network learned about occlusions, light and shading - all through self-supervision. #CogSci2018
Botvinick: Another example: Machine theory of mind. Have the AI system predict what an agent is about to do, given side information about what this agent has done in other situations. #CogSci2018
Botvinick: "Theory of mind" paper: Given that the agent has sought out orange objects in the past, I predict it will seek out this other orange object in the present scene. #CogSci2018
Botvinick: So far this sounds like the triumph of nurture, Empiricism and Locke. But there's really is an important role for nature in the results I've discussed. #CogSci2018
Botvinick: For example, the best systems trained on ImageNet are based on convolutional neural networks: feature detectors have to be the same across the entire picture. This is useful because of the structure of the data: translation invariance. #CogSci2018
Botvinick: The systems that are "killing it" in AI are NOT general-purpose systems: they've been hobbled to focus on particular solutions to the problem, that reflect structure in the world. #CogSci2018
Botvinick: The bias-variance tradeoff: narrow the hypothesis space and provide sufficient inductive bias, but still give the system enough flexibility to fit the data. #CogSci2018
Botvinick: Recurrent neural network capitalize on time invariance: what's useful at one time is likely to be useful at a different time, so it makes sense to share the weight across time points. #CogSci2018
Botvinick: Another form of reusability in the environment (a paper written by Peter Battaglia and @jhamrick): relations between objects. #CogSci2018
Botvinick: But you need to know a lot about the structure of the domain to figure out what the relevant entities and relationship between those entities are. Our work only tells the system that there ARE objects and relations, and have it learn what those are. #CogSci2018
Botvinick: Visual question answering system. The question is fed through an RNN and the image through a CNN. Then representations of chunks of an image are fed into a relational network; the same weights (representing relations) are applied to all pairs of chunks. #CogSci2018
Botvinick: @jhamrick has a poster on relational inductive bias - poster 104 today (Friday). #CogSci2018
Botvinick: Deep reinforcement learning systems need thousands of hours to learn what a human (specifically, @LakeBrenden) can learn very quickly. Clearly, we need stronger inductive biases. Can we learn these biases without knowledge engineering? #CogSci2018
Botvinitck: The solution is "learning to learn", based on Harlow's monkey experiments (in the 1940s). #CogSci2018
Botvinick: This meta-learning or learning-to-learn effect arises naturally in a reinforcement learning system that is based on a recurrent neural network. (Wang et al 2018, Nature Neuroscience.) #CogSci2018
Botvinick: Two forms of memory in a neural network: its weights and its activity. Slow learning through weights absorbs the general principle that underlie multiple tasks, such that the activity of the system can solve the immediate task very quickly. #CogSci2018
Botvinick: @chelseabfinn has a different approach to meta-learning. Our recurrent approach is arguably more similar to what happens in the prefrontal cortex. Prediction errors in our system mirror prediction errors in the dopamine system. #CogSci2018
Botvinick: Meta-learning becomes even more interesting when you bring in evolution. Evolution is a learning algorithm that extracts structure from the data and builds in biases that are useful for learning in an individual. #CogSci2018
Botvinick: Example of synthetic evolution research: "Meta-Learning by the Baldwin Effect", Fernando et al arxiv.org/abs/1806.07917
Botvinick: All of this is a victory for empiricism in the sense that the biases are learned; but this is a different sense of empiricism, because they're learned over multiple time scales (in evolution or within an individual). #CogSci2018
Botvinick: An important aspect of AI research is task design: we need to design our tasks such that they give our agents the opportunity to learn what they need to learn. This isn't the same as knowledge engineering, but is also a thorny problem. #CogSci2018
Botvinick: Another challenge is to understand how neural networks do what they do. This is another area (in additional to model design and task design) where psychologists and neuroscientists can help. #CogSci2018
Botvinick: Exchanges between psychology, neuroscience and AI have always been a part of the discipline of cognitive science. But now is a golden age for these exchanges. #CogSci2018
Botvinick: My experience is that these exchanges are best when we have psychologists and neuroscientists at the table when we develop an AI system. The ideas can't be packaged by psychologists and shipped over to the AI world. #CogSci2018
