Mucho: A Timeline-Based Immersive Tool for Prototyping Interactive Multimodal XR Experiences
Open Access
- 27 June 2025
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 9 (4) , 1-20
Abstract
Prototyping multimodal interaction in XR (Augmented/Virtual Reality) is challenging for non-technical designers. Moreover, existing XR prototyping tools do not support concurrent multimodal interaction and lack an overall view of the interaction design of the experience. We created Mucho , a novel no-coding immersive tool for prototyping interactive multimodal XR experiences. Mucho supports three modes: In recording, designers can demonstrate input examples to populate a timeline representation with events. In playback, designers can create a video prototype based on the timeline while adding system actions at the corresponding timeframes. In live mode, Mucho creates a state machine to test the interactive experience in runtime. We present an evaluation by demonstration to illustrate how Mucho can prototype a diverse set of examples including modalities such as hand gestures, proximity, speech, and gaze. Finally, we discuss the limitations of our approach and outline future directions to support immersive prototyping of multimodal XR experiences.Keywords
Funding Information
- Aarhus University Research Foundation (AUFF-F-2021-7-2)
This publication has 48 references indexed in Scilit:
- Proxemic-aware Augmented Reality For Human-Robot Interaction2023
- Auditory stimuli degrade visual performance in virtual realityScientific Reports, 2020
- MRCAT: In Situ Prototyping of Interactive AR Environments2020
- The Trouble with Augmented Reality/Virtual Reality Authoring Tools2018
- Exploring the Benefits of Immersive End User Development for Virtual RealityLecture Notes in Computer Science, 2016
- A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment2015
- A usability study of multimodal input in an augmented reality environmentVirtual Reality, 2013
- Immersive authoring of Tangible Augmented Reality content: A user studyJournal of Visual Languages & Computing, 2009
- SwingStates: adding state machines to Java and the Swing toolkitSoftware: Practice and Experience, 2007
- A Multimodal Interface Framework for Using Hand Gestures and Speech in Virtual Environment ApplicationsLecture Notes in Computer Science, 1999