Multimodal Interfaces for Effective Teleoperation
Preprint
Abstract
Research in multi-modal interfaces aims to provide solutions to immersion and increase overall human performance. A promising direction is combining auditory, visual and haptic interaction between the user and the simulated environment. However, no extensive comparisons exist to show how combining audiovisuohaptic interfaces affects human perception reflected on task performance. Our paper explores this idea. We present a thorough, full-factorial comparison of how all combinations of audio, visual and haptic interfaces affect performance during manipulation. We evaluate how each interface combination affects performance in a study (N=25) consisting of manipulating tasks of varying difficulty. Performance is assessed using both subjective, assessing cognitive workload and system usability, and objective measurements, incorporating time and spatial accuracy-based metrics. Results show that regardless of task complexity, using stereoscopic-vision with the VRHMD increased performance across all measurements by 40% compared to monocular-vision from the display monitor. Using haptic feedback improved outcomes by 10% and auditory feedback accounted for approximately 5% improvement.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- Haptic feedback combined with movement sonification using a friction sound improves task performance in a virtual throwing taskJournal on Multimodal User Interfaces, 2018
- Stereoscopic 3D displays and human performance: A comprehensive reviewDisplays, 2014
- Immersive 3-D teleoperation of a search and rescue robot using a head-mounted display2009
- Measuring and defining the experience of immersion in gamesInternational Journal of Human-Computer Studies, 2008
- Is neocortex essentially multisensory?Trends in Cognitive Sciences, 2006
- Feeling with the mind's eye: contribution of visual cortex to tactile perceptionBehavioural Brain Research, 2002
- Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: Normative data, reliability, and validityArchives of Physical Medicine and Rehabilitation, 2002
- Haptic study of three-dimensional objects activates extrastriate visual areasNeuropsychologia, 2002
- Visuo-haptic object-related activation in the ventral visual pathwayNature Neuroscience, 2001
- Multimodal virtual reality: Input‐output devices, system integration, and human factorsInternational Journal of Human–Computer Interaction, 1996