VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models
Top Cited Papers
Open Access
- 11 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-20
Abstract
Tour guidance in virtual museums encourages multi-modal interactions to boost user experiences, concerning engagement, immersion, and spatial awareness. Nevertheless, achieving the goal is challenging due to the complexity of comprehending diverse user needs and accommodating personalized user preferences. Informed by a formative study that characterizes guidance-seeking contexts, we establish a multi-modal interaction design framework for virtual tour guidance. We then design VirtuWander, a two-stage innovative system using domain-oriented large language models to transform user inquiries into diverse guidance-seeking contexts and facilitate multi-modal interactions. The feasibility and versatility of VirtuWander are demonstrated with virtual guiding examples that encompass various touring scenarios and cater to personalized preferences. We further evaluate VirtuWander through a user study within an immersive simulated museum. The results suggest that our system enhances engaging virtual tour experiences through personalized communication and knowledgeable assistance, indicating its potential for expanding into real-world scenarios.Keywords
All Related Versions
Funding Information
- Guangzhou Basic and Applied Basic Research Foundation (2023A03J0142, 2024A04J6462)
This publication has 52 references indexed in Scilit:
- Analysis of virtual museums in terms of design and perception of presenceEducation and Information Technologies, 2023
- An Approach to Facilitate Visitors’ Engagement with Contemporary Art in a Virtual Museum2022
- Redefining the Digital Paradigm for Virtual Museums2021
- Virtual Museum ‘Takeouts’ and DIY Exhibitions–Augmented Reality Apps for Scholarship, Citizen Science and Public Engagement2021
- Avatars as storytellers: affective narratives in virtual museumsPersonal and Ubiquitous Computing, 2020
- Enhancing Emotional Effectiveness of Virtual-Reality Experiences with Voice Control Interfaces2019
- Human-Computer Interactions in Museums2019
- The development and evaluation of a survey to measure user engagementJournal of the American Society for Information Science and Technology, 2009
- Living in a Learning Society2006
- A methodology for the evaluation of travel techniques for immersive virtual environmentsVirtual Reality, 1998