Diego Gonzalez Morin

Novel and cutting-edge use cases have arisen since the first deployments of the fifth generation (5G) of mobile communications. We can already find multiple 5G simulators and emulators in the state-of-the-art which allow engineers and researchers to thoroughly study and test the network. However, the 5G ecosystem is not only limited to the network itself, a fast development of 5G-specific use cases can considerably accelerate the development of telecommunication technologies. We present FikoRE, our open-source real-time Radio Access Networks (RAN) emulator designed for application-level experimentation and prototyping, which aims to boost the development of novel use cases, such as eXtended Reality (XR) and the recently announced Metaverse. Its modularity and straightforward structure allow multidisciplinary users to rapidly utilize or even modify it to test their own applications. Besides, our thoughtful simplifications and assumptions allow to handle actual IP traffic above 1 Gb/s from multiple sources in real time, all while simulating hundreds of virtual User Equipments (UEs), outperforming other relevant cutting-edge RAN emulators. We thoroughly describe these modeling assumptions that allow high scalability and performance while preserving the overall accuracy. The high IP traffic throughput and multi-user capabilities allow FikoRE to be used in realistic advanced scenarios as XR offloading or multi-user XR. Finally, we present a set of validation results showing the emulation capabilities and modeling accuracy of the 5G full stack.

In this work, we explore the creation of self-avatars through video pass-through in mixed reality (MR) applications. We present our end-to-end system, including custom MR video pass-through implementation on a commercial head-mounted display (HMD), our deep learning-based real-time egocentric body segmentation algorithm, and our optimized offloading architecture, to communicate the segmentation server with the HMD. To validate this technology, we designed an immersive VR experience where the user has to walk through a narrow tile path over an active volcano crater. The study was performed under three-body representation conditions: virtual hands, video pass-through with color-based full-body segmentation, and video pass-through with deep learning full-body segmentation. This immersive experience was carried out by 30 women and 28 men. To the best of our knowledge, this is the first user study focused on evaluating video-based self-avatars to represent the user in a MR scene. Results showed no significant differences between the different body representations in terms of presence, with moderate improvements in some Embodiment components between the virtual hands and full-body representations. Visual Quality results showed better results from the deep-learning algorithms in terms of the whole body perception and overall segmentation quality. In this study, we provide some discussion regarding the use of video-based self-avatars and some reflections on the evaluation methodology. The proposed E2E solution is in the boundary of the state-of-the-art, so there is still room for improvement before it reaches maturity. However, this solution serves as a crucial starting point for MR applications where users can feel immersed and interact with their own bodies.