Speech/gesture interface to a visual computing environment for molecular biologists
Authors:
- T.S. HuangTHT.S. Huang
- V.I. PavlovicVPV.I. Pavlovic
- Yunxin ZhaoYZYunxin Zhao
- Zion LoZLZion Lo
- S. ChuSCS. Chu
- K. SchulKSK. Schul
- 1 January 1996
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 3 (10514651) , 964-968 vol.3
Abstract
Recent progress in 3-D, immersive display and virtual reality (VR) technologies has made possible many exciting applications, for example interactive visualization of complex scientific data. To fully exploit this potential there is a need for "natural" interfaces that allow the manipulation of such displays without cumbersome attachments. In this paper we describe the use of visual hand gesture analysis and speech recognition for developing a speech/gesture interface for controlling a 3-D display. The interface enhances an existing application, VMD, which is a VR visual computing environment far molecular biologists. The free hand gestures are used for manipulating the 3-D graphical display together with a set of speech commands. We describe the visual gesture analysis and the speech analysis techniques used in developing this interface. The dual modality of speech/gesture is found to greatly aid the interaction capability.Keywords
This publication has 5 references indexed in Scilit:
- VMD: Visual molecular dynamicsJournal of Molecular Graphics, 1996
- A Multimodal framework for Interacting with Virtual Environments1996
- Gestures with speech for graphic manipulationInternational Journal of Man-Machine Studies, 1993
- Automatic recognition of keywords in unconstrained speech using hidden Markov modelsIEEE Transactions on Acoustics, Speech, and Signal Processing, 1990
- Image analysis via the general theory of moments*Journal of the Optical Society of America, 1980