Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Christopher Manning Profile picture
@chrmanning
Jul 26, 2022 • 3 tweets • 2 min read • Read on X
I finally read @boazbaraktcs’s blog on DL vs Stats.
A great mind-clearing read! 👍
“Yes, that was how we thought about NNs losing out due to bias/variance in ~2000”
“Yes, pre-trained models really are different to classical stats, even if math is the same”
windowsontheory.org/2022/06/20/the…
A bit more nuance could be added to this 2nd para on Supervised Learning. Initial breakthroughs _were_ made in #NLProc via unsupervised learning prior to AlexNet—the word vectors of Collobert&Weston (2008/2011) like related stuff in RBMs, Google cat, etc. jmlr.org/papers/volume1… Image
But, for a few years, the siren song of the effectiveness of end-to-end deep learning on large supervised datasets was irresistible and very successful, probably partly because of how, in the over-parameterized regime, it does do representation learning as this post argues.

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Christopher Manning

Christopher Manning Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @chrmanning

Christopher Manning Profile picture
Dec 2, 2022
Dear @emilymbender—and @Abebab—you need to keep “reminding” people of your viewpoint because it is not an argument that is convincing to all or a self-evident truth. It is a particular academic position, which lots of people support but a good number of others disagree with. 1/8
The position of Bender & @alkoller is that of 20th century formal semantics, based on Anglo-American philosophy of language & formal grammar, the approach we were both brought up on. However, it had already been rejected by Ludwig Wittgenstein in 1953: topologicalmedialab.net/xinwei/classes… 2/
“You say: the point isn't the word, but its meaning, and you think of the meaning as a thing of the same kind as the word, though also different from the word. Here the word, there the meaning. The money, and the cow that you can buy with it. But contrast: money, and its use.” 3/
Read 8 tweets
Christopher Manning Profile picture
Sep 21, 2022
I’m happy to share the published version of our ConVIRT algorithm, appearing in #MLHC2022 (PMLR 182). In 2020, this was a pioneering work in contrastive learning of perception by using naturally occurring paired text. Unfortunately, things took a winding path from there. 🧵👇 Image
The paper (Contrastive Learning of Medical Visual Representations from Paired Images and Text, @yuhaozhangx @hjian42 Yasuhide Miura @chrmanning & @curtlanglotz) shows much better unsupervised visual representation learning using paired text versus vision alone (SimCLR, MoCo v2) ImageImage
However, sometimes you don’t get lucky with conference reviewing—even when at a highly privileged institution. We couldn’t interest reviewers at ICLR2020 or ICCV2021. I think the fact that we showed gains in radiology (x-rays) not general vision seemed to dampen interest….
Read 7 tweets
Christopher Manning Profile picture
Jul 30, 2022
I would suggest that this thread errs by over-representing the proportion of the time in which human “reasoning” is actually anything akin to mathematical reasoning, such as the example of solving SAT instances. 1/
To start with a Go analogy: A moderately skilled player can exhaustively read out a 6–10 move life-and-death problem or end game sequence—or usually work it out more quickly using pattern-based shortcuts! But for longer, more complex things such as fuseki (opening) sequences, 2/
they “reason” about moves—“it would be better for me to approach here than to extend on the other side, because then they would jump and I could then approach strengthening my corner while attacking”—but really this is pattern matching! It’s not like solving a SAT problem. 3/
Read 6 tweets
Christopher Manning Profile picture
Sep 12, 2021
It’s great getting to read my colleagues @robreich, @mehran_sahami & Jeremy Weinstein’s book, System Error. Building a broad understanding of problems with big tech and techno utopianism is such an important topic for this decade.
harpercollins.com/products/syste…
Some thoughts below. 🧵👇 Image
The authors rightly stress many key problems that have emerged: deficiencies of simplistic metrics, dangers of tech monopolies, balancing innovation against status as a public utility, what should become of privacy and free speech in a world of corporate-owned public squares?
However, they end up questioning an “optimization mindset” in general, and I’m not sure that’s right. There are many ways that optimization can go wrong, which they discuss: people can adopt simplistic uni-dimensional metrics (e.g., “connecting more people is good for all”) or …
Read 18 tweets
Christopher Manning Profile picture
Dec 31, 2020
@yoavgo @ChrisGPotts I take primary blame for advocating the anonymity period. It was an honest attempt at a compromise middle ground. With the passage of time, I admit that it seems a bit flawed, as more people aim for “the anonymity period deadline” but the real question is what would be better?
@yoavgo @ChrisGPotts Your suggestion, @yoavgo, is to move the dial all the way to the left or to the right, but such extremist positions seldom are optimal in a complex and varied world. We could survey again, but I suspect the situation is similar to where it was 3 years ago: one large group, ...
@yoavgo @ChrisGPotts which seemed to center on white, male, Americans were all for preprints (mostly killing anonymous reviewing) while another large group (mainly people outside the above group) believed strongly in preserving anonymous reviewing. Should we just fully go with one group? ...
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(