Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Behnam Neyshabur Profile picture
@bneyshabur
Apr 30, 2021 • 7 tweets • 10 min read • Read on X
Scrolly
Come to our talks and posters at #ICLR2021 to discuss our findings on understanding and improving deep learning! Talks and posters are available now! Links to the talks, posters, papers and codes in the thread:

1/7
When Do Curricula Work? (Oral at #ICLR2021)
with @XiaoxiaWShirley and @ethansdyer

Paper: openreview.net/forum?id=tW4QE…
Code: github.com/google-researc…
Video and Poster: iclr.cc/virtual/2021/p…

2/7
Sharpness-Aware Minimization for Efficiently Improving Generalization (Spotlight at #ICLR2021 )
with @Foret_p, Ariel Kleiber and @TheGradient

Paper: openreview.net/forum?id=6Tm1m…
Code: github.com/google-researc…
Video and Poster: iclr.cc/virtual/2021/p…

3/7
Understanding the Failure Modes of Out-of-Distribution Generalization (Poster at #ICLR2021 )
with @_vaishnavh and @AJAndreassen

Paper: openreview.net/forum?id=fSTD6…
Code: github.com/google-researc…
Video and Poster: iclr.cc/virtual/2021/p…

4/7
The Deep Bootstrap: Good Online Learners are Good Offline Generalizers (Poster at #ICLR2021 )
with @PreetumNakkiran and @HanieSedghi

Paper: openreview.net/forum?id=guetr…
Code: github.com/preetum/deep-b…
Video and Poster: iclr.cc/virtual/2021/p…

5/7
Are wider nets better given the same number of parameters? (Poster at #ICLR2021 )
with @_anna_go and @guygr

Paper: openreview.net/forum?id=_zx8O…
Code: github.com/google-researc…
Video and Poster: iclr.cc/virtual/2021/p…

6/7
Extreme Memorization via Scale of Initialization (Poster at #ICLR2021 )
with @n0royalroad and @AshokCutkosky

Paper: openreview.net/forum?id=Z4R1v…
Code: github.com/google-researc…
Video and Poster: iclr.cc/virtual/2021/p…

7/7

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Behnam Neyshabur

Behnam Neyshabur Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @bneyshabur

Behnam Neyshabur Profile picture
Feb 5
I've left Anthropic to start something new.
🧵 Image
Working at Anthropic was a wonderful experience.

Extremely high talent density, amazing culture, mission-driven, zero politics, leadership with real technical depth. Over the past year, I’ve learned so much about what made Anthropic successful and developed great respect for the founders and the team. I'm grateful to have been part of such an extraordinary organization.

Reflecting on the last 20 years, I see three phases—and I'm now entering a fourth:
1) Confusion
As an undergrad at Sharif University, I wasn't excited by the traditional academic path, and was confused about what I wanted to do with my life. Later, I decided to immigrate to the US, and the only viable path that covered living costs was a PhD program. So that's what I did! Here’s the entire story:
Read 6 tweets
Behnam Neyshabur Profile picture
Dec 21, 2022
These days, many people are interested in getting a PhD in ML. I think you should think really hard before committing to a PhD program in ML. Why?

I'm going to summarize some thoughts in this thread:

1/10
Graduate degree in ML is overrated. So is having publications in top ML venues. One can accomplish a lot in this field without any of these. The truth is that you don’t need to cover a lot of background before you can do interesting things in ML.

2/10
The ML field continues to become more and more accessible everyday. Everything you need to learn is available online. There is a lot of push to make ML methods/models open-source and reproducible. Many people are also producing useful educational content.

3/10
Read 10 tweets
Behnam Neyshabur Profile picture
Jun 18, 2021
🆕 📰: Deep Learning Through the Lens of Example Difficulty

We introduce a measure of computational difficulty and show its surprising relationships with different deep learning phenomena.

Paper: arxiv.org/abs/2106.09647

with @Robert_Baldock & Hartmut Maennel

1/ Image
✅ We introduce a measure of computational example difficulty: the prediction depth (PD). PD is the earliest layer after which the network’s final prediction is already determined.

✅ We use k-NN classifier probes to determine the prediction of each layer (left panel).

2/ Image
✅ Prediction depth is higher for examples and datasets that seem more difficult (Fig. 1).

✅ PD is consistent across random seeds and similar architectures (Fig. 2).

3/ Image
Read 11 tweets
Behnam Neyshabur Profile picture
Jan 13, 2021
Some people say that one shouldn't care about publication and the quality matters. However, the job market punishes those who don’t have publications in top ML venues. I empathize with students and newcomers to ML whose good papers are not getting accepted. #ICLR2021
1/
Long thread at the risk of being judged:

I just realized that in the last 6 years, 21 of my 24 papers have been accepted to top ML conf in their FIRST submission even though the majority of them were hastily-written borderline papers (not proud of this). How is this possible?
2/
At this point, I'm convinced that this cannot be explained by a combination of luck and quality of the papers. My belief is that the current system has lots of unnecessary and sometimes harmful biases which is #unfair to new comers and anyone who is outside of the "norm".
3/
Read 17 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(