An Audio-Ultrasound Synchronized Database of Tongue Movement for Mandarin speech
Open Access
- 11 April 2025
- journal article
- dataset
- Published by Springer Nature in Scientific Data
- Vol. 12 (1) , 1-8
Abstract
Ultrasound imaging has been widely adopted in speech research to visualize dynamic tongue movements during speech production. These images are universally used as visual feedback in interventions for articulation disorders or visual cues in speech recognition. Nevertheless, the availability of high-quality audio-ultrasound datasets remains scarce. The present study, therefore, aims to construct a multimodal database designed for Mandarin speech. The dataset integrates synchronized ultrasound images of lingual movement, and the corresponding audio recordings and text annotations elicited from 43 healthy speakers and 11 patients with dysarthria through speech tasks (including vowels, monosyllables, and sentences), with a total duration of 22.31 hours. In addition, a customized helmet structure was employed to stabilize the ultrasound probe, precisely controlling for head movement and minimizing displacement interference, The proposed database carries apparent values in automatic speech recognition, silent interface development, and research in speech pathology and linguistics.Keywords
This publication has 28 references indexed in Scilit:
- An initial framework for use of ultrasound by speech and language therapists in the UK: Scope of practice, education and governanceUltrasound, 2022
- Multimodal dataset of real-time 2D and static 3D MRI of healthy French speakersScientific Data, 2021
- A multispeaker dataset of raw and reconstructed speech production real-time MRI video and 3D volumetric imagesScientific Data, 2021
- Extending Ultrasound Tongue Shape Complexity Measures to Speech Development and DisordersJournal of Speech, Language, and Hearing Research, 2021
- Sibilant Fricative Merging in Taiwan Mandarin: An Investigation of Tongue Postures using Ultrasound ImagingLanguage and Speech, 2019
- Tongue Part Movement Trajectories for /r/ Using UltrasoundPerspectives of the Asha Special Interest Groups, 2019
- Tongue Shape Dynamics in Swallowing Using Sagittal UltrasoundDysphagia, 2018
- Acoustic and tongue kinematic vowel space in speakers with and without dysarthriaInternational Journal of Speech-Language Pathology, 2016
- Real-time magnetic resonance imaging and electromagnetic articulography database for speech production research (TC)The Journal of the Acoustical Society of America, 2014
- Comparing tongue shapes from ultrasound imaging using smoothing spline analysis of varianceThe Journal of the Acoustical Society of America, 2006