Audiovisual corpus to analyze whisper speech
Authors:
- 1 May 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8101-8105
Abstract
Current automatic speech recognition (ASR) systems cannot recognize whisper speech with high accuracy. ASR systems are trained with neutral speech, which have significant acoustic differences with whisper speech (i.e., energy, duration, harmonics structure, and spectral slope). Given the limitations of speech-based systems to process whisper speech, we propose to explore the benefits of visual features describing the orofacial area. We hypothesize that the lips' articulation between whisper and neutral speech is similar, providing a valuable whisper-invariant modality. This paper introduces the first audiovisual corpus of whisper speech. While we are targeting over 40 speakers, the current corpus has recordings from eleven subjects who were asked to read TIMIT sentences, and isolated digits alternating between neutral and whisper speech. The corpus also includes spontaneous recordings, in which the subject answered a series of general questions. The paper also analyzes an exhaustive set of audiovisual features, including action units (AUs), lip spreading, fundamental frequency, intensity, MFCCs, and formants. We study the differences in the features' distributions between whisper and neutral speech using Kullback-Leibler divergence (KLD). Then, we conducted statistical test to determine whether the differences in the features are statistically significant. The results support our hypothesis that visual features are less affected by whisper speech.Keywords
This publication has 11 references indexed in Scilit:
- A Comprehensive Vowel Space for Whispered SpeechJournal of Voice, 2012
- Opensmile2010
- Acoustic analysis for speaker identification of whispered speech2010
- Acoustic Analysis of Consonants in Whispered SpeechJournal of Voice, 2008
- Whispering Speaker Identification2007
- Head Movement Synthesis Based on Semantic and Prosodic Features for a Chinese Expressive Avatar2007
- Automatic Recognition of Facial Actions in Spontaneous ExpressionsJournal of Multimedia, 2006
- Analysis and recognition of whispered speechSpeech Communication, 2005
- Asynchrony modeling for audio-visual speech recognition2002
- Formant-Frequency Differences Between Isolated Whispered and Phonated Vowel Samples Produced by Adult Female SubjectsJournal of Speech, Language, and Hearing Research, 1984