Direct articulatory observation reveals phoneme recognition performance characteristics of a self-supervised speech model
- 1 November 2024
- journal article
- research article
- Published by Acoustical Society of America (ASA) in JASA Express Letters
- Vol. 4 (11)
Abstract
Variability in speech pronunciation is widely observed across different linguistic backgrounds, which impacts modern automatic speech recognition performance. Here, we evaluate the performance of a self-supervised speech model in phoneme recognition using direct articulatory evidence. Findings indicate significant differences in phoneme recognition, especially in front vowels, between American English and Indian English speakers. To gain a deeper understanding of these differences, we conduct real-time MRI-based articulatory analysis, revealing distinct velar region patterns during the production of specific front vowels. This underscores the need to deepen the scientific understanding of self-supervised speech model variances to advance robust and inclusive speech technology.Keywords
Funding Information
- National Science Foundation (NSF IIS 2311676)
This publication has 12 references indexed in Scilit:
- Articulatory Representation Learning via Joint Factor Analysis and Neural Matrix Factorization2023
- Unsupervised Cross-Lingual Representation Learning for Speech Recognition2021
- A multispeaker dataset of raw and reconstructed speech production real-time MRI video and 3D volumetric imagesScientific Data, 2021
- Real Time Speech Enhancement in the Waveform Domain2020
- User Experience with Smart Voice Assistants: The Accent Perspective2019
- Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi2017
- Vocal tract cross-distance estimation from real-time MRI using region-of-interest analysis2013
- Articulation rate across dialect, age, and genderLanguage Variation and Change, 2009
- Region Segmentation in the Frequency Domain Applied to Upper Airway Real-Time Magnetic Resonance ImagesIEEE Transactions on Medical Imaging, 2008
- Relations of sex and dialect to reductionSpeech Communication, 1994