Yongkook Won

This study examines the validity of WER as a proxy for pronunciation quality in EFL contexts. Human ratings of comprehensibility and accentedness were compared with WER and automated pronunciation scores from six ASR systems — Kaldi, wav2vec 2.0, HuBERT, Whisper (Base and Large-v3), and Microsoft Azure — using 190 read-aloud recordings by Korean elementary learners. With respect to pronunciation scoring, Azure’s phoneme-level accuracy scores demonstrated moderate correlations with human judgments, while Kaldi’s GOP scores showed no meaningful association. Analysis of WER revealed a critical trade-off between ASR accuracy and perceptual sensitivity: high-performing systems such as Whisper Large-v3 and Azure produced near-zero WERs but weakly correlated with human ratings. In contrast, mid-performing systems such as Whisper Base and HuBERT showed stronger correlations, indicating that moderate WER values may better reflect pronunciation variation. These results underscore the limitations of WER in advanced ASR systems and the need for perceptually grounded, interpretable metrics.

The primary objective of this study is to identify the types of errors made by Korean college students in an oral proficiency interview in relation to specific task topics, and to examine how these errors affect their lexico-grammatical proficiency scores. Ninety-six two-minute-long audio clips of 32 Korean college students on three different topics were transcribed. Lexico-grammatical errors were then coded for statistical analysis and lexico-grammatical scores were estimated using many-facet Rasch measurement analysis with two raters. Friedman tests and Wilcoxon signed-rank tests showed that noun phrase, verb phrase, and prepositional phrase errors were more frequently found with the descriptive tasks than compare-and-contrast or hypothetical prompts. A hierarchical multiple regression analysis showed that noun phrase and verb phrase errors accounted for 22% of the variance in lexico-grammatical scores. Adding utterance length variables to the initial regression model explained an additional 43% of the variance in the lexico-grammatical scores. These findings suggest that noun phrase errors and verb phrase errors should be a priority in English classes, and that it is beneficial to teach English speaking skills in a way that takes into account the task characteristics and contextual factors.