Florian Brühlmann

Virtual reality (VR) applications are increasingly permeating our lives. The immersion provided by VR enables novel interactions with data that would be impossible in conventional environments. Especially regarding multimedia data, VR could overcome existing limitations when browsing videos to find specific scenes. This paper introduces a virtual multimedia drawer, tailored to VR environments, to enable novel ways to interact with videos. A within-subjects design experiment (N=24) was conducted to evaluate the multimedia drawer on user experience, efficiency, and effectiveness. Results show that the multimedia drawer, while taking slightly longer to locate a particular scene within a video for certain types of tasks (i.e., sequence tasks), provides statistically significantly higher levels of enjoyment, novelty, and stimulation and is preferred over conventional timeline-based approaches. Implications of quantitative and qualitative results regarding the design and features of the multimedia drawer as a video browsing method in VR are critically discussed.

Measuring the subjective experience of digital game players is essential to player experience research. Recently, the Player Experience Inventory (PXI) was developed, which assesses both functional and psychosocial consequences of digital gameplay. We present a pre-registered independent online study with a large sample to provide additional evidence of psychometric quality for the PXI. Responses from 1518 participants were collected, rating a recent or memorable experience playing a digital game using the PXI and related measures. While our results from standard psychometric reliability and validity analyses generally favored the PXI, we also identified challenges with the immersion construct. Further, we find a ten-factor model, or alternatively, an 11-factor should enjoyment be measured, to fit our collected data best. In sum, the PXI is a valuable tool to measure a variety of constructs central to player experience.

Carelessness or insufficient effort responding is a widespread problem in online research, with estimates ranging from 3% to almost 50% of participants in online surveys being inattentive. While detecting carelessness has been subject to multiple studies, the factors that reduce or prevent carelessness are not as well understood. Initial evidence suggests that warning statements prior to study participation may reduce carelessness, but there is a lack of conclusive high-powered studies. This preregistered randomized controlled experiment aimed to test the effectiveness of a warning statement and an improved implementation of a warning statement in reducing participant inattention. A study with 812 participants recruited on Amazon Mechanical Turk was conducted. Results suggest that presenting a warning statement is not effective in reducing carelessness. However, requiring participants to actively type the warning statement statistically significantly reduced carelessness as measured with self-reported diligence, even-odd consistency, psychometric synonyms and antonyms, and individual response variability. The active warning statements also led to statistically significantly more attrition and potentially deterred those who were likely to be careless from even participating in this study. We show that the current standard practice of implementing warning statements is ineffective and novel methods to prevent and deter carelessness are needed.

Past research has demonstrated that aesthetics affect users' experiences in various ways. However, there is little research on the impact of interface aesthetics on user performance in a smartphone app context. The present paper addresses this research gap using an online experiment (N = 281). Two variants of the same web app were created and manipulated in their aesthetics. Participants were randomly assigned to either variant and asked to explore the app before answering questions concerning the app's content. Results showed a significant positive effect of aesthetics on perceived usability and aesthetics. Furthermore, results point toward a positive impact of interface aesthetics on performance (i.e., the number of questions answered correctly). Thus, results indicate that a visually appealing smartphone web app increases users' subjective experience and objective performance compared to an unaesthetic app. This suggests that user interface aesthetics impact users' experiences and provide stakeholders with quantifiable value and competitive advantage.

Auditing plays a pivotal role in the development of trustworthy AI. However, current research primarily focuses on creating auditable AI documentation, which is intended for regulators and experts rather than end-users affected by AI decisions. How to communicate to members of the public that an AI has been audited and considered trustworthy remains an open challenge. This study empirically investigated certification labels as a promising solution. Through interviews (N = 12) and a census-representative survey (N = 302), we investigated end-users’ attitudes toward certification labels and their effectiveness in communicating trustworthiness in low- and high-stakes AI scenarios. Based on the survey results, we demonstrate that labels can significantly increase end-users’ trust and willingness to use AI in both low- and high-stakes scenarios. However, end-users’ preferences for certification labels and their effect on trust and willingness to use AI were more pronounced in high-stake scenarios. Qualitative content analysis of the interviews revealed opportunities and limitations of certification labels, as well as facilitators and inhibitors for the effective use of labels in the context of AI. For example, while certification labels can mitigate data-related concerns expressed by end-users (e.g., privacy and data protection), other concerns (e.g., model performance) are more challenging to address. Our study provides valuable insights and recommendations for designing and implementing certification labels as a promising constituent within the trustworthy AI ecosystem.

Trust is crucial for human interaction with artificial intelligence (AI) and is frequently measured through questionnaires or rating scales. One commonly used questionnaire in AI research is the Trust between People and Automation scale (TPA). However, its psychometric quality has yet to be examined in the context of AI. More recently, a Trust Scale for Explainable AI (TXAI) was recommended but not empirically evaluated. In this study, we assessed the psychometric qualities of both scales, using confirmatory and exploratory factor analyses to test the scales’ validity and coefficients α and ω for reliability estimation. Our results suggested good psychometric quality for the TXAI after removing one item. Concerning the TPA, acceptable quality was only achieved when using a two-factor model (trust and distrust) and after removing two items. We provide recommendations for using the two scales and evidence to distinguish trust and distrust as separate psychological constructs.

Motives are essential concepts in understanding a player’s experience in video games. We report and describe the analysis of a widely used questionnaire to measure players’ motives in video games, the Motives for Online Gaming Questionnaire (MOGQ). The present research aimed to investigate the psychometric quality of the MOGQ in the context of League of Legends (LoL). The MOGQ is a 27-item self-report scale designed to measure the motives for playing online games. To this end, 256 participants completed an online survey asking about their experiences and motives to play LoL. Results of confirmatory and exploratory factor analyses indicate weaknesses in the original 7-factor model. By removing five conspicuous items from the original 7-factor model, we propose an alternative 22-item version of the MOGQ. Additional confirmatory and exploratory factor analyses results indicate that the 22-item version of the MOGQ is more suitable in the context of LoL than the original 27-item questionnaire. However, further investigation into the quality of this alternative version compared to the original questionnaire is needed.

Flow is a highly influential concept across many research domains. Because of the close links between enjoyment and flow, it is also a central concept in game research. Accordingly, many measurement instruments in the form of questionnaires have been developed to measure flow in video games. However, limited independent validation work has been carried out to date. In this project, we evaluated the recently developed Video Game Dispositional Flow Scale (VGDFS) in the context of League of Legends. An online survey posted on the online community reddit.com yielded a sample of N = 140. Confirmatory factor analysis did not support the originally proposed structure. Subsequent exploratory factor analysis resulted in inconclusive findings. Convergent validity with other questionnaires for the measurement of interest, enjoyment, and flow could be demonstrated in our data, although with unusually high correlations between the VGDFS and another flow measure. For divergent validity, correlations with questionnaires for the measurement of boredom and trait anxiety in the hypothesized direction could be found. Findings suggest that the VGDFS needs further evaluation in different video game contexts while also hinting at measurement problems possibly related to conceptual issues with flow.