These days, many people are interested in getting a PhD in ML. I think you should think really hard before committing to a PhD program in ML. Why?

I'm going to summarize some thoughts in this thread:

1/10

https://twitter.com/ylecun/status/1605450677806895104

Graduate degree in ML is overrated. So is having publications in top ML venues. One can accomplish a lot in this field without any of these. The truth is that you don’t need to cover a lot of background before you can do interesting things in ML.

2/10

🆕 📰: Deep Learning Through the Lens of Example Difficulty

We introduce a measure of computational difficulty and show its surprising relationships with different deep learning phenomena.

Paper: arxiv.org/abs/2106.09647

with @Robert_Baldock & Hartmut Maennel

1/ ✅ We introduce a measure of computational example difficulty: the prediction depth (PD). PD is the earliest layer after which the network’s final prediction is already determined.

✅ We use k-NN classifier probes to determine the prediction of each layer (left panel).

2/

Come to our talks and posters at #ICLR2021 to discuss our findings on understanding and improving deep learning! Talks and posters are available now! Links to the talks, posters, papers and codes in the thread:

1/7 When Do Curricula Work? (Oral at #ICLR2021)
with @XiaoxiaWShirley and @ethansdyer

Paper: openreview.net/forum?id=tW4QE…
Code: github.com/google-researc…
Video and Poster: iclr.cc/virtual/2021/p…

2/7

Some people say that one shouldn't care about publication and the quality matters. However, the job market punishes those who don’t have publications in top ML venues. I empathize with students and newcomers to ML whose good papers are not getting accepted. #ICLR2021
1/ Long thread at the risk of being judged:

I just realized that in the last 6 years, 21 of my 24 papers have been accepted to top ML conf in their FIRST submission even though the majority of them were hastily-written borderline papers (not proud of this). How is this possible?
2/