Jack Jansen

Recent advancements in immersive communication technologies have facilitated the exploration of novel modalities of social interaction, utilizing platforms that offer a spectrum of representations from simplified avatars to photorealistic volumetric and point-cloud reconstructions. This study focused on how these different representations and degrees of immersion impact communication processes and collaborative task performance. The primary objective was to examine potential variances in performance and interaction quality across different systems during a basic communication task. To achieve this, we employed Meta Horizon Workrooms as a three degrees of freedom (3DoF) system, VR2Gather as a six degrees of freedom (6DoF) system, and the MS Teams meeting application as a comparative benchmark. A user study was conducted whereby participants engaged in a charade game, alternating between the roles of “mimic” and “guesser.” Throughout the experimental session, data were collected to assess comfort, presence, task load, and interaction quality, with performance quantified by the number of words guessed per minute. The findings indicated no significant differences in workload, presence, or simulator sickness across platforms; however, significant differences were observed in audio–visual quality, communication effort, and enjoyment. Performance metrics revealed that participants exhibited the highest task performance within MS Teams, followed by VR2Gather, and subsequently Horizon Workrooms. These findings underscore the influence of embodiment fidelity and system familiarity on communication quality within immersive environments. We discuss system-level constraints and propose considerations for the design of future social extended reality (XR) platforms that effectively balance realism with usability and performance.

Professional collaboration in XR production requires aligning spatial vision between stakeholders with different roles and expertise. While producers understand XR design affordances, clients often lack experiential knowledge, creating communication gaps during early project discussions. Conventional 2D meeting platforms cannot adequately support discussing inherently spatial XR concepts. We explore how Social XR addresses this challenge by situating both parties within a shared three-dimensional meeting environment. Through an in-the-wild study with XR producers in Denmark and museum curators in the Netherlands (N=8), we examine how shared immersive space shapes pre-production communication. Our findings demonstrate Social XR’s value for improving communication compared to conventional meetings: clients gain direct spatial understanding of XR possibilities, while producers can observe client reactions and guide discussions in real time. The study also reveals divergent meeting intentions between producers and clients - a dynamic invisible in 2D contexts. Additionally, we discuss implications for designing and evaluating collaborative XR meeting environments where success depends on aligning creative vision rather than completing defined tasks.

Immersive technologies promise to revolutionize communication through enhanced sense of presence and interactivity. To enable interaction, reliable low-latency transport mechanisms are needed to handle the large volumes of data created by complex 3D objects. In this paper, we propose an open-source, codec-independent, selective forwarding unit (SFU) for real-time volumetric video streaming using WebRTC. For evaluation purposes, we provide a reference client implementation by extending VR2Gather, a TCP-based system for immersive communication. We conduct extensive evaluations using both new and existing datasets to compare the performance of WebRTC against TCP-based protocols in an emulated testbed environment. The evaluations demonstrate that WebRTC outperforms other protocols in high-latency scenarios and adapts video quality to user movement 13% and 36% faster than its TCP-based counterparts in networks with 5 ms and 10 ms of network latency, respectively.

Perceptual quality assessment of Dynamic Point Cloud (DPC) contents plays an important role in various Virtual Reality (VR) applications that involve human beings as the end user. Understanding and modeling perceptual quality assessment is greatly enriched by insights from visual attention. However, incorporating aspects of visual attention in DPC quality models is largely unexplored, as ground-truth visual attention data is scarcely available. Besides, testing methods and procedures for collecting visual attention data are still to be agreed on. This paper presents a dataset containing subjective opinion scores and visual attention maps of DPCs, collected in a VR environment using eye-tracking technology. Both the quality score and eye-tracking data were collected during a subjective quality assessment experiment, in which subjects were instructed to watch and rate DPCs at various degradation levels under 6 Degrees-of-Freedom (DoF) inspection, using a head-mounted display. Qualitative interview analysis was also conducted after the experiment. The dataset consists of 50 DPCs, including 5 reference DPCs, with each reference encoded at 3 distortion levels using 3 different codecs (namely G-PCC, V-PCC, CWI-PCL), amounting to a total of 9 degraded version per reference. Additionally, it incorporates 1,000 gaze trials from 40 participants, yielding a total of 15,000 visual attention maps across all the DPCs. We additionally benchmark objective quality metrics originally designed for static point clouds, evaluating their performance in our dataset using two temporal pooling strategies. Furthermore, we employ the visual attention data that are retrieved during our experiment to evaluate whether the performance of widely used objective quality metrics is improved by considering subjective measurements of visual attention. This dataset establishes a link between quality assessment and visual attention within the context of DPC. Moreover, thematic analysis of the interviews helps uncover user behavior and factors impacting perceptual quality for DPC in 6 DoF. This work deepens our understanding of DPC quality assessment and visual attention, driving progress in the realm of VR experiences and perception.

Social Virtual Reality is envisioned to transform how individuals communicate remotely, offering a sense of immersion and co-presence within a virtual space. Current platforms enabling remote social interactions rely on synthetic user representations. We address this limitation by enabling realistic human representation through volumetric content capture, encoding and transmission. Specifically, we present an extended version of VR2Gather, now a fully open source Unity package, available at https://github.com/cwi-dis/VR2Gather-acmmm-oss. Our platform is a customisable system to transmit volumetric content in a multi-party real-time environment, easy to integrate into existing applications.

Immersive technologies like eXtended Reality (XR) are the next step in videoconferencing. In this context, understanding the effect of delay on communication is crucial. This article presents the first study on the impact of delay on collaborative tasks using a realistic Social XR system. Specifically, we design an experiment and evaluate the impact of end-to-end delays of 300, 600, 900, 1,200, and 1,500 ms on the execution of a standardized task involving the collaboration of two remote users that meet in a virtual space and construct block-based shapes. To measure the impact of the delay in this communication scenario, objective and subjective data were collected. As objective data, we measured the time required to execute the tasks and computed conversational characteristics by analyzing the recorded audio signals. As subjective data, a questionnaire was prepared and completed by every user to evaluate different factors such as overall quality, perception of delay, annoyance using the system, level of presence, cybersickness, and other subjective factors associated with social interaction. The results show a clear influence of the delay on the perceived quality and a significant negative effect as the delay increases. Specifically, the results indicate that the acceptable threshold for end-to-end delay should not exceed 900 ms. This article additionally provides guidelines for developing standardized XR tasks for assessing interaction in Social XR environments.

Virtual Reality telecommunication systems promise to overcome the limitations of current real-time teleconferencing solutions, by enabling a better sense of immersion and fostering more natural interpersonal interactions. Many solutions that currently enable immersive teleconferencing employ synthetic avatars to represent their users. However, photorealistic reconstructions have been shown to increase the sense of presence with respect to synthetic avatars in tele-immersive scenarios. In this paper, we present VR2Gather, a costumizable end-to-end system to transmit volumetric contents in multi-party real-time communication. We present the architecture and evaluate the costs and benefits of using different modules and transport mechanisms in terms of CPU usage, latency, and bandwidth. Moreover, we report the user experience based on applications the system has been used for, and how it was customised to meet the requirements using different acquisition and rendering modules.