Yang Wei

Mispronunciation detection (MD) is a key component in computer-assisted pronunciation training (CAPT) and speaking tests. Most MD systems adopt a production view, measuring phone-level deviation from a canonical pronunciation (Native Norm) or the expected pronunciation of a target population (Target Norm). Yet, pronunciation assessment is fundamentally perceptual: listeners map speech to linguistic categories under uncertainty and with individual psychological priors, so judgments are inherently subjective and lack a single gold standard. Labels are therefore often aggregated (e.g., voting), but aggregation rules are themselves subjective, require many annotators, and entangle individual perception with social consensus, complicating model training. In this paper, we propose a “Perception Norm”, which models MD as the decision process of individual annotators and trains models to simulate single listeners rather than an annotator pool. To support this study, we introduce UY/CH-CHILD-MA, a corpus of Uyghur-accented child Mandarin words and phrases with four independent phone-level annotations. Our experiments reveal substantial inter-annotator variation and show that a Transformer with pre-training and fine-tuning can learn annotator-specific patterns with high accuracy. Finally, we present a committee ensemble that combines annotator models using application-matched aggregation rules to produce task-specific assessments. The data and source code will be made publicly available upon publication.

Pronunciation assessment is central to computer-assisted pronunciation training (CAPT) and speaking tests, yet most systems still adopt a native norm, treating deviations from canonical L1 pronunciations as errors. In contrast, rating rubrics and psycholinguistic evidence emphasize intelligibility for a target listener population and show that listeners rapidly adapt their phonetic categories to new accents. We argue that automatic assessment should likewise be referenced to the target learner group. We build a Transformer-based mispronunciation detection (MD) model that computationally mimics listener adaptation: it is first pre-trained on multi-speaker Librispeech, then fine-tuned on the non-native L2-ARCTIC corpus that represents a specific learner population. Fine-tuning, using either synthetic or human MD labels, constrains updates to the phonetic space (i.e., the representation space used to encode phone-level distinctions, the learned phone/phonetic embedding space, and its alignment with acoustic representations), which means that only the phonetic module is updated while the rest of the model stays fixed. Relative to the pre-trained model, L2 adaptation substantially improves MD recall and F1, increasing ROC–AUC from 0.72 to 0.85. The results support a target-population norm and inform the design of perception-aligned, fairer automatic pronunciation assessment systems.