Thread by @PetarV_93

The crowd has spoken! 🙃 A thread with early-stage machine learning research advice follows below. 👇🧵

Important disclaimer before proceeding: these are my personal views only, and likely strongly biased by my experiences and temperament. Hopefully useful nonetheless! 1/15

https://twitter.com/PetarV_93/status/1385932158599114752

During the early stages of my PhD, one problem would often arise: I would come up with ideas that simply weren't the right kind of idea for the kind of hardware/software/expertise setup I had in my department. 2/15

This would lead me on 'witch hunts' that took months (sometimes forcing me to spend my own salary on compute!). Game-changer for me was corresponding w/ researchers that are influential to the work I'd like to do: first learn from their perspectives, eventually internships. 3/15

My internships @Mila_Quebec (developing GAT/DGI) both started w/ emails to Yoshua Bengio, and in both cases we initially discussed something completely unrelated to what ended up those two papers. (first time, it was multimodal computer vision; second time, consciousness). 4/15

Similarly, my internship at @BellLabs (developing X-CNN/X-LSTM; with @niclane7) also started with an email thread, aimed at low-resource machine learning, totally unrelated to computer vision. 5/15

Internships (/short research placements) are a great way to get focused up towards solving a particular problem (or set of problems): it's a pre-determined amount of time, usually working alongside a domain expert (so they'll know what is novel, exciting and feasible),... 6/15

...and usually it exposes you to a wider group of researchers who can give invaluable context/connections that simply wouldn't've been possible elsewhere. Actually, lessons learnt at internships can also positively feedback to your workflow outside them. 7/15

Eventually, with enough experience you'll absorb the field, and you'll have a much better feel for what is feasible. One skill I've gradually learnt in this regard is, once you have a hypothesis you'd like to test, to always try to "hardcode" the problem as much as possible. 8/15

By this I mean, for example, directly feeding to the model data it is not supposed to have, to the point where if the model doesn't work in this setting, you (usually) either have a bug somewhere or the hypothesis is very likely false. 9/15

Once your model fits the hardcoded problem correctly, you can gradually relax the 'restricted' data you feed it, making sure it still works every step of the way.

This was an impactful workflow that proved critical in many of my recent algorithmic reasoning papers. 10/15

I think it's a bit of a sad reality that students are driven to "publish or perish". If you work on problems that you find highly meaningful, with a niche that will find your solutions valuable, I don't find it an absolute requirement that the work must be published. 11/15

But one piece of advice I'd give is, once you have a meaningful hypothesis and maybe 1-2 small experiments verifying it in certain conditions, to always submit your work to a relevant **workshop**. 12/15

Workshops typically have a much higher (~75+%) acceptance rate, are a great platform to get feedback on your work before it is publication-ready, and---somewhat ironically---the fact that they are attached to a major conference means you still get a lot of exposure. 13/15

In fact, because workshops often tackle a narrow 'niche' of researchers, it's very likely that you'll get concentrated feedback exactly from people who could end up your future reviewers or even collaborators. :) 14/15

That's it for now! Hope you found it useful -- and any feedback / personal experiences welcome. :)

Tagging @HennyJieCC @jeongwhan_choi @SMRhadou @shuosuree @g_sokar @josephimperial_ @o_v_shake @k_saifullaah @nicola_decao @akbirkhan as you've explicitly registered interest. 15/15