Eyes on the Narrative: Exploring the Impact of Visual Realism and Audio Presentation on Gaze Behavior in AR Storytelling
Authors:
- Florian WeidnerFWFlorian Weidner
- Jakob HartbrichJHJakob Hartbrich
- Stephanie Arevalo ArboledaSAStephanie Arevalo Arboleda
- Christian KunertCKChristian Kunert
- Christian SchneiderwindCSChristian Schneiderwind
- Chenyao DiaoCDChenyao Diao
- Christoph GerhardtCGChristoph Gerhardt
- Tatiana SurduTSTatiana Surdu
- Wolfgang BrollWBWolfgang Broll
- Stephan WernerSWStephan Werner
- Alexander RaakeARAlexander Raake
Open Access
- 4 June 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Abstract
Augmented Reality (AR) and Virtual Reality (VR) are essential tools for researchers and practitioners, serving purposes from training to entertainment: many of these applications rely on agents. This study explores the impact of agent characteristics on user reactions, focusing on gaze as a primary visual attention indicator in AR and VR. While existing research has investigated the agent’s gaze and its influence on the user, it is unclear how the agent’s auralization and visualization influence gaze behaviour. We investigate this by studying the impact of rendering style and type of audio on gaze behaviour during a narrative AR experience. Participants listened to a story with the agent visualized as a cartoon-style or realistic virtual human and auralized with spatial or non-spatial audio. The results revealed that the agent’s rendering style significantly influenced gaze behaviour, with cartoon-style agents capturing more visual attention. Audio variations did not yield significant differences. Together, our findings inform the design of AR user interfaces with agents, suggesting that low-realism visualizations are more captivating and, thus, more suitable for experiences where the user is supposed to look at the storyteller.Keywords
Funding Information
- Carl-Zeiss-Foundation (CO-HUMANICS)
- German Federal Ministry of Education and Research (BMBF) (16SV8922)
- European Research Council (101021229)
This publication has 39 references indexed in Scilit:
- Are you talking to me? An Audio Augmented Reality conversational guide for cultural heritagePervasive and Mobile Computing, 2023
- Mouth-in-the-Door: The Effect of a Sound Image of an Avatar Intruding on Personal Space That Deviates in Position From the Visual ImageIEEE Access, 2022
- Validating the effects of immersion and spatial audio using novel continuous biometric sensor measures for Virtual Reality2022
- Effects of Rendering Styles of a Virtual Character on Avoidance Movement Behavior2022
- Sympathy for the digital: Influence of synthetic voice on affinity, social presence and empathy for photorealistic virtual humansComputers & Graphics, 2022
- Uncanny valley hypothesis and hierarchy of facial features in the human likeness continua: An eye-tracking approach.Psychology & Neuroscience, 2022
- Examining the Use of Nonverbal Communication in Virtual AgentsInternational Journal of Human–Computer Interaction, 2021
- Using Facial Animation to Increase the Enfacement Illusion and Avatar Self-IdentificationIEEE Transactions on Visualization and Computer Graphics, 2020
- Live Speech Driven Head-and-Eye Motion GeneratorsIEEE Transactions on Visualization and Computer Graphics, 2012
- Gaze, conversational agents and face-to-face communicationSpeech Communication, 2010