Text Entry in Virtual Environments using Speech and a Midair Keyboard
Authors:
- 22 March 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 27 (5) , 2648-2658
Abstract
Entering text in virtual environments can be challenging, especially without auxiliary input devices. We investigate text input in virtual reality using hand-tracking and speech. Our system visualizes users' hands in the virtual environment, allowing typing on an auto-correcting midair keyboard. It also supports speaking a sentence and then correcting errors by selecting alternative words proposed by a speech recognizer. We conducted a user study in which participants wrote sentences with and without speech. Using only the keyboard, users wrote at 11 words-per-minute at a 1.2% error rate. Speaking and correcting sentences was faster and more accurate at 28 words-per-minute and a 0.5% error rate. Participants achieved this performance despite half of sentences containing an uncommon out-of-vocabulary word (e.g. proper name). For sentences with only in-vocabulary words, performance using speech and midair keyboard corrections was faster at 36 words-per-minute with a low 0.3% error rate.Keywords
Funding Information
- NSF (IIS-1750193)
This publication has 52 references indexed in Scilit:
- PalmType2015
- A Dose of Reality2015
- Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry2015
- VelociTap2015
- Selection-Based Mid-Air Text Entry on Large Displays2013
- The word-gesture keyboardCommunications of the ACM, 2012
- Text Input Techniques for Immersive Virtual Environments: An Empirical ComparisonProceedings of the Human Factors and Ergonomics Society Annual Meeting, 2002
- Multimodal error correction for speech user interfacesACM Transactions on Computer-Human Interaction, 2001
- Finding consensus in speech recognition: word error minimization and other applications of confusion networksComputer Speech & Language, 2000
- Taming recognition errors with a multimodal interfaceCommunications of the ACM, 2000