Talk two of paper session 15 at #FAccT21:
"Spoken Corpora Data, Automatic Speech Recognition, and Bias Against African American Language: The Case of Habitual ‘Be’" by Joshua Martin
"Spoken Corpora Data, Automatic Speech Recognition, and Bias Against African American Language: The Case of Habitual ‘Be’" by Joshua Martin
Studies have begun to be published about racial linguistic bias, but still not too many. Point to ASR (automatic speech recognition) systems and how a paper came out last year about how 5 major systems (Apple, Amazon, Google, ..) had much higher error rates for Black speakers
This paper specifically looks at the specific case of (the habitual) "be" that is unique to AAVE/AAL; "Angela be studying" is used as an example, pointing out that "Angela is studying" is *not* an adequate or correct equivalent. Speech recognition systems struggle with this.
ASR systems, due to their training, might take "Angela be studying" and recognize it instead with 93% accuracy as "Angela is studying", when this is an incorrect translation. Therefore these ASR systems have bias that works against AAVE/AAL speakers.
They looked at 4 common ASR corpora (some which notably only have 4% Black speakers included) and the Corpus of Regional African American Language (CORAAL) (which has 100% Black speakers included) to compare the differences in the frequency and usage of the habitual 'be'.
Shocking results: LibriSpeech (134M words) only has 42 instances of the habitual 'be'. Switchboard (3M words) only has *3* instances. Whereas CORAAL (1M words) has 485 instances. These other corpora are the ones used for ASR systems, but are woefully lacking in 'be' instances.
53% of CORAAL texts include the habitual 'be', whereas for these 4 other popular corpora, only 0% - 2% of their texts include the habitual 'be'. The findings reveal a lack of representation in these datasets, thus affecting the ASRs as well. Yikes!! 😬
• • •
Missing some Tweet in this thread? You can try to
force a refresh
