Profile picture
, 3 tweets, 1 min read
My Authors
Read all threads
Self-Supervised Learning making strides in speech recognition.

Wav2Vec 2.0 from FAIR uses a kind of contrastive SSL for pre-training.

This is the first time an SSL system reaches the very best results on a number of different ASR tasks.

arxiv.org/abs/2006.11477
1/N
After SSL pre-training, a word error rate of 10% can be obtained with just 10 minutes of labeled training data.

With only 1h of labeled data, this beats the best previous SSL method trained on 100h of labeled data.
2/N
Wav2vec 2.0 matches the best known word error rate when trained on the full 960h of labeled data in LibriSpeech, but with a rather simpler architecture.
3/N
 N=3
Missing some Tweet in this thread? You can try to force a refresh.

Keep Current with Yann LeCun

Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

More from @ylecun see all

Profile picture
Yann LeCun
@ylecun
I'm an immigrant.

I came first to work at Bell Labs on a J-1 visa, because I thought I'd stay only a year or two.
But I stayed longer and got an H1-B visa.
Then I got a green card....
1/N
nytimes.com/2020/06/22/us/…
I hesitated to take up citizenship during the GW Bush years, waiting for the country to become respectable again.
But after Bush's re-election, I just wanted to be able to vote and kick out the neocon bastards.
So I became a citizen just in time to vote for Barack Obama.
2/N
As an immigrant, scientist, academic, liberal, atheist, and Frenchman, I am a concentrate of everything the American Right hates.
3/N
Read 8 tweets
Profile picture
Yann LeCun
@ylecun
@timnitGebru If I had wanted to "reduce harms caused by ML to dataset bias", I would have said "ML systems are biased *only* when data is biased".
But I'm absolutely *not* making that reduction.
1/N
@timnitGebru I'm making the point that in the *particular* *case* of *this* *specific* *work*, the bias clearly comes from the data.
2/N
@timnitGebru There are many causes for *societal* bias in ML systems
(not talking about the more general inductive bias here).
1. the data, how it's collected and formatted.
2. the features, how they are designed
3. the architecture of the model
4. the objective function
5. how it's deployed
Read 17 tweets
Profile picture
Yann LeCun
@ylecun
We often hear that AI systems must provide explanations and establish causal relationships, particularly for life-critical applications.
Yes, that can be useful. Or at least reassuring....
1/n
But sometimes people have accurate models of a phenomenon without any intuitive explanation or causation that provides an accurate picture of the situation. In many cases of physical phenomena, "explanations" contain causal loops where A causes B and B causes A.
2/n
A good example is how a wing causes lift. The computational fluid dynamics model, based on Navier-Stokes equations, works just fine. But there is no completely-accurate intuitive "explanation" of why airplanes fly.
3/n
Read 9 tweets

Embed code for your website

×
Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!