,
8 tweets,
2 min read
Read on Twitter
How can you do great AI research when you don't have access to google-scale compute? By being weird. The big tech companies are obsessed with staying nimble despite being big, and some succeed to some extent. But they can't afford to be as weird as a lone looney professor.
A lone professor with a handful of students and a few computers can never win over @DeepMindAI or @facebookai in a straight competition. But we can afford to try methods that make absolutely no sense, or attack problems that nobody wants to solve as they don't look like problems.
To the extent I've done anything useful or worthwhile in my career, it's always been through trying to solve a problem nobody thought of, or trying a method that shouldn't work. Very often the useful/publishable end result was nothing like what I thought I was working towards.
So go on, be weird. Out-weird the giants. Even if they're both nimble and powerful, they cannot be as stupid and ridiculous as you. Because how would that look? To managers, investors, board members, the general public? You can afford to completely disregard such entities.
Now, I'm not saying that there's no value in throwing giant compute resources at some problem, and trying to break a long-standing benchmark. That's all good, I'm happy that there are people that do those things. But I'm happy that I don't have to do it. Because it's a bit boring
And of course the advantage of the big tech companies is not only in having many GPUs. It's also in having large teams of highly competent people working on the project non-stop without having to e.g. teach or go to faculty meetings. Still, you can do it.
Many of the best ideas still come from academia, even though the best results don't.
See also:
