The crowd has spoken! 🙃 A thread with early-stage machine learning research advice follows below. 👇🧵

Important disclaimer before proceeding: these are my personal views only, and likely strongly biased by my experiences and temperament. Hopefully useful nonetheless! 1/15
During the early stages of my PhD, one problem would often arise: I would come up with ideas that simply weren't the right kind of idea for the kind of hardware/software/expertise setup I had in my department. 2/15
This would lead me on 'witch hunts' that took months (sometimes forcing me to spend my own salary on compute!). Game-changer for me was corresponding w/ researchers that are influential to the work I'd like to do: first learn from their perspectives, eventually internships. 3/15
My internships @Mila_Quebec (developing GAT/DGI) both started w/ emails to Yoshua Bengio, and in both cases we initially discussed something completely unrelated to what ended up those two papers. (first time, it was multimodal computer vision; second time, consciousness). 4/15
Similarly, my internship at @BellLabs (developing X-CNN/X-LSTM; with @niclane7) also started with an email thread, aimed at low-resource machine learning, totally unrelated to computer vision. 5/15
Internships (/short research placements) are a great way to get focused up towards solving a particular problem (or set of problems): it's a pre-determined amount of time, usually working alongside a domain expert (so they'll know what is novel, exciting and feasible),... 6/15
...and usually it exposes you to a wider group of researchers who can give invaluable context/connections that simply wouldn't've been possible elsewhere. Actually, lessons learnt at internships can also positively feedback to your workflow outside them. 7/15
Eventually, with enough experience you'll absorb the field, and you'll have a much better feel for what is feasible. One skill I've gradually learnt in this regard is, once you have a hypothesis you'd like to test, to always try to "hardcode" the problem as much as possible. 8/15
By this I mean, for example, directly feeding to the model data it is not supposed to have, to the point where if the model doesn't work in this setting, you (usually) either have a bug somewhere or the hypothesis is very likely false. 9/15
Once your model fits the hardcoded problem correctly, you can gradually relax the 'restricted' data you feed it, making sure it still works every step of the way.

This was an impactful workflow that proved critical in many of my recent algorithmic reasoning papers. 10/15
I think it's a bit of a sad reality that students are driven to "publish or perish". If you work on problems that you find highly meaningful, with a niche that will find your solutions valuable, I don't find it an absolute requirement that the work must be published. 11/15
But one piece of advice I'd give is, once you have a meaningful hypothesis and maybe 1-2 small experiments verifying it in certain conditions, to always submit your work to a relevant **workshop**. 12/15
Workshops typically have a much higher (~75+%) acceptance rate, are a great platform to get feedback on your work before it is publication-ready, and---somewhat ironically---the fact that they are attached to a major conference means you still get a lot of exposure. 13/15
In fact, because workshops often tackle a narrow 'niche' of researchers, it's very likely that you'll get concentrated feedback exactly from people who could end up your future reviewers or even collaborators. :) 14/15
That's it for now! Hope you found it useful -- and any feedback / personal experiences welcome. :)

Tagging @HennyJieCC @jeongwhan_choi @SMRhadou @shuosuree @g_sokar @josephimperial_ @o_v_shake @k_saifullaah @nicola_decao @akbirkhan as you've explicitly registered interest. 15/15

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Petar Veličković

Petar Veličković Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @PetarV_93

16 Nov 20
Over the past weeks, several people have reached out to me for comment on "Combining Label Propagation and Simple Models Out-performs Graph Neural Networks" -- a very cool LabelProp-based baseline for graph representation learning. Here's a thread 👇 1/14
Firstly, I'd like to note that, in my opinion, this is a very strong and important work for representation learning on graphs. It provides us with so many lightweight baselines that often perform amazingly well -- on that, I strongly congratulate the authors! 2/14
I think most of the discussion comes from the title -- most people reaching out to me ask "Does this mean we don't need GNNs at all?", "Have GNNs been buried?", etc.

In reality, this work reinforces something we've known in graph representation learning for quite some time. 3/14
Read 14 tweets
17 Sep 20
As requested , here are a few non-exhaustive resources I'd recommend for getting started with Graph Neural Nets (GNNs), depending on what flavour of learning suits you best.

Covering blogs, talks, deep-dives, feeds, data, repositories, books and university courses! A thread 👇
For blogs, I'd recommend:
- @thomaskipf's post on Graph Convolutional Networks:…
- My blog on Graph Attention Networks:
- A series of comprehensive deep-dives from @mmbronstein:…
For a comprehensive overview of the area in the form of a talk, I would highly recommend @xbresson's guest lecture at NYU's Deep Learning course:

Read 8 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!