Zero-Shot Automatic Pronunciation Assessment
Preprint
Abstract
Automatic Pronunciation Assessment (APA) is vital for computer-assisted language learning. Prior methods rely on annotated speech-text data to train Automatic Speech Recognition (ASR) models or speech-score data to train regression models. In this work, we propose a novel zero-shot APA method based on the pre-trained acoustic model, HuBERT. Our method involves encoding speech input and corrupting them via a masking module. We then employ the Transformer encoder and apply k-means clustering to obtain token sequences. Finally, a scoring module is designed to measure the number of wrongly recovered tokens. Experimental results on speechocean762 demonstrate that the proposed method achieves comparable performance to supervised regression baselines and outperforms non-regression baselines in terms of Pearson Correlation Coefficient (PCC). Additionally, we analyze how masking strategies affect the performance of APA.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden UnitsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Deep Feature Transfer Learning for Automatic Pronunciation Assessment2021
- Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing2021
- speechocean762: An Open-Source Non-Native English Speech Corpus for Pronunciation Assessment2021
- The Zero Resource Speech Challenge 2021: Spoken Language Modelling2021
- Computer-Assisted Pronunciation Training (CAPT): Current Issues and Future DirectionsRELC Journal, 2021
- Automatic Scoring at Multi-Granularity for L2 Pronunciation2020
- An Improved Goodness of Pronunciation (GoP) Measure for Pronunciation Evaluation with DNN-HMM System Considering HMM Transition Probabilities2019
- Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiersSpeech Communication, 2015
- An overview of spoken language technology for educationSpeech Communication, 2009