VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models
Authors:
Preprint
Abstract
Tour guidance in virtual museums encourages multi-modal interactions to boost user experiences, concerning engagement, immersion, and spatial awareness. Nevertheless, achieving the goal is challenging due to the complexity of comprehending diverse user needs and accommodating personalized user preferences. Informed by a formative study that characterizes guidance-seeking contexts, we establish a multi-modal interaction design framework for virtual tour guidance. We then design VirtuWander, a two-stage innovative system using domain-oriented large language models to transform user inquiries into diverse guidance-seeking contexts and facilitate multi-modal interactions. The feasibility and versatility of VirtuWander are demonstrated with virtual guiding examples that encompass various touring scenarios and cater to personalized preferences. We further evaluate VirtuWander through a user study within an immersive simulated museum. The results suggest that our system enhances engaging virtual tour experiences through personalized communication and knowledgeable assistance, indicating its potential for expanding into real-world scenarios.Keywords
All Related Versions
- Version 1, 2024-01-22, ArXiv
- Version 2, 2024-01-23, ArXiv
- Published version: , 1.
This publication has 48 references indexed in Scilit:
- Analysis of virtual museums in terms of design and perception of presenceEducation and Information Technologies, 2023
- Evaluation of voice commands for mode change in virtual reality implant planning procedureInternational Journal of Computer Assisted Radiology and Surgery, 2022
- Value-based model of user interaction design for virtual museumCCF Transactions on Pervasive Computing and Interaction, 2021
- Avatars as storytellers: affective narratives in virtual museumsPersonal and Ubiquitous Computing, 2020
- Enhancing Emotional Effectiveness of Virtual-Reality Experiences with Voice Control Interfaces2019
- Exploring the relationship between presence and enjoyment in a virtual museumInternational Journal of Human-Computer Studies, 2010
- The development and evaluation of a survey to measure user engagementJournal of the American Society for Information Science and Technology, 2009
- Virtual museums, a survey and some issues for considerationJournal of Cultural Heritage, 2009
- A platform for virtual museums with personalized contentMultimedia Tools and Applications, 2008
- A methodology for the evaluation of travel techniques for immersive virtual environmentsVirtual Reality, 1998