,
5 tweets,
2 min read
Read on Twitter
I've been thinking a bit about the growing practice of fine-tuning generic pretrained models: first in computer vision, now NLP (highly recommend @seb_ruder's great article on this ruder.io/nlp-imagenet/)...Last time I mentioned this, people were skeptical that RL would be next.
E.g. folks pointed out that RL is maybe the wrong level/domain of analysis, and maybe there is insufficient commonality across RL tasks people care about for this to make sense, etc... But in any case, I suspect more will happen in this vein beyond vision and NLP.
The simplest extrapolation would just be that, just as vision and NLP gradually matured to a point of being super commercially/creatively/etc. relevant over the course of many years, multimodal stuff combining vision+NLP might eventually reach that point + follow a similar path.
I'm not sure what that might look like, though. There is probably also low hanging fruit in fine-tuning from generative models trained on various modalities. Will be very interesting to see how the science, economics, art, etc. of all of this plays out in the coming years! /Fin
P.S. Did not mean to imply RL *won't* be next, just that it's not my focus here :) I think the jury is still out on how progress in sample efficient RL/meta-learning specifically, simulators, transfer learning, research/industry preference for different tasks, etc. will play out.
