What should a meta-learner optimize? What if we make it chase its own future outputs?

Turns out, it can improve meta-optimization, set new SOTAs, and lead to new types of meta-learning. arxiv.org/pdf/2109.04504…

w. Y. Schroecker, @tomzhavy, @hado, D. Silver, S. Singh.
🧵👇 The meta-learner's goal is to match a target. That gives us a lot of flexibility: the matching function can ease optimization and we can get open-ended meta-learning by generating targets from the meta-learner itself: the better the meta-learner gets, the better the targets 🚀