EchoWhisper
Authors:
Open Access
- 4 September 2020
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 4 (3) , 1-27
Abstract
With the rapid growth of artificial intelligence and mobile computing, intelligent speech interface has recently become one of the prevalent trends and has already presented huge potentials to the public. To address the privacy leakage issue during the speech interaction or accommodate some special demands, silent speech interfaces have been proposed to enable people's communication without vocalizing their sound (e.g., lip reading, tongue tracking). However, most existing silent speech mechanisms require either background illuminations or additional wearable devices. In this study, we propose the EchoWhisper as a novel user-friendly, smartphone-based silent speech interface. The proposed technique takes advantage of the micro-Doppler effect of the acoustic wave resulting from mouth and tongue movements and assesses the acoustic features of beamformed reflected echoes captured by the dual microphones in the smartphone. Using human subjects who perform a daily conversation task with over 45 different words, our system can achieve a WER (word error rate) of 8.33%, which shows the effectiveness of inferring silent speech content. Moreover, EchoWhisper has also demonstrated its reliability and robustness to a variety of configuration settings and environmental factors, such as smartphone orientations and distances, ambient noises, body motions, and so on.Keywords
This publication has 41 references indexed in Scilit:
- BodyScope2012
- Effects of Sound Pressure Levels and Sensitivity to Noise on Mood and Behavioral Intent in a Controlled Fine Dining Restaurant EnvironmentJournal of Culinary Science & Technology, 2010
- Silent-speech enhancement using body-conducted vocal-tract resonance signalsSpeech Communication, 2010
- Micro-doppler effect in radar: phenomenon, model, and simulation studyIEEE Transactions on Aerospace and Electronic Systems, 2006
- The statistical modeling of road traffic noise in an urban settingCities, 2003
- Acoustic noise and echo cancelling with microphone arrayIEEE Transactions on Vehicular Technology, 1999
- Active noise control: a tutorial reviewProceedings of the IEEE, 1999
- Broadband DOA estimation using frequency invariant beamformingIEEE Transactions on Signal Processing, 1998
- Multiple emitter location and signal parameter estimationIEEE Transactions on Antennas and Propagation, 1986
- Some Airflow, Volume, and Duration Characteristics of Oral ReadingJournal of Speech and Hearing Research, 1978