Super-excited about our new #ICASSP2020 paper on "Universal Phone Recognition with a Multilingual Allophone System" arxiv.org/abs/2002.11800
We create a multi-lingual ASR model that can do zero-shot phone recognition in up to 2,186 languages! How? A little linguistics :) 1/5
We create a multi-lingual ASR model that can do zero-shot phone recognition in up to 2,186 languages! How? A little linguistics :) 1/5
In our speech there are phonemes (sounds that can support lexical contrasts in a *particular* language) and their corresponding phones (the sounds that are actually spoken, which are language *independent*). Most multilingual ASR models conflate these two concepts. 2/5 
We create a model that first recognizes to language-independent phones, and then converts these phones to language-specific phonemes. This makes our underlying representations of phones more universal and generalizable across languages. 3/5
We create a DB of phone-phoneme mappings, AlloVera that contains these mappings for several languages: github.com/dmort27/allove…
We also demonstrate that we can use PHOIBLE, a large phonetic DB with 2,186 languages to restrict phones in new languages: phoible.org 4/5
We also demonstrate that we can use PHOIBLE, a large phonetic DB with 2,186 languages to restrict phones in new languages: phoible.org 4/5
Results show that these methods work, helping both more standard multilingual recognition, and recognition on entirely new languages.
Next, we're interested in incorporating these into real systems for low- or zero-resource ASR. Please contact us if you're interested! 5/5
Next, we're interested in incorporating these into real systems for low- or zero-resource ASR. Please contact us if you're interested! 5/5
Thanks to the *many* people who supported the project, including the participants at the LTLDR 2019 workshop (sites.google.com/view/ltldr/home), and most of all Xinjian, our fearless leader who actually got the project done.
