Mario Botsch

This paper presents the first systematic investigation of how congruence in visual self-representation influences self-perception and behavior. We span a continuum from the physical self through avatars with graded self-similarity to clearly dissimilar avatars in virtual reality (VR). In a 1x4 within-user study, participants completed movement and quiz tasks in either physical reality or a digital twin environment in VR, where they embodied one of three avatars: a photorealistic self-similar avatar, a dissimilar same-gender avatar, or a dissimilar opposite-gender avatar. Subjective measures included presence, sense of embodiment, self-identification, and perceived change, and were complemented by an objective movement metric of behavioral change. Compared to physical reality, VR, even with a self-similar avatar, produced lower presence, a weaker sense of embodiment, and reduced self-identification, revealing a persistent gap in visual congruence. Within VR, self-similar avatars enhanced body ownership, self-location, and self-identification relative to dissimilar avatars. Conversely, dissimilar avatars produced measurable behavioral changes compared with self-similar ones. Gender cues, however, had little impact in gender-neutral tasks. Overall, the findings show that photorealistic self-similar avatars reinforce embodiment and self-identification. However, VR still falls short of achieving congruence with physical reality, underscoring key challenges for avatar realism and ecological validity.

Most real‐time animation techniques for digital humans are limited to deforming the outer skin surface. Geometric skinning methods are highly efficient but struggle with artifacts such as collapsing joints or self‐intersections when animating inner anatomy along with the outer skin. Volumetric physics‐based simulations, on the other hand, naturally resolve these issues by coordinating bones, muscles, and skin, but are far too slow for interactive use. We solve this problem by training a neural network to predict deformation gradients. Learning deformation gradients instead of vertex displacements makes our method naturally robust to artifacts such as element inversion or volume deviation. Our model, trained on high‐quality finite element simulations, generalizes well across diverse body shapes and poses. This enables anatomically consistent and physically grounded animation of bones, muscles, and skin at interactive frame rates.

Every human body is different, however, current movement analysis does not reflect that, as it heavily relies on generic musculoskeletal models. Usually, these models are scaled to match the participants’ body segment lengths and body weight, but not taking individual body shape into account. This can lead to errors in the estimation of joint forces and torques, which are important to accurately estimate musculoskeletal variables. Thus, we developed a method to estimate body segment inertial parameters based on body hulls acquired via smartphone pictures. From the body hull, we infer the skeletal shape and pose, and then estimate the distribution of bone, lean, and fatty tissues. We then segment the body hull and assign each tissue type a density, which is used to calculate the body segment inertial parameters. To allow for the use of our method with existing data, we also introduce two new generic musculoskeletal models, which are based on the average standing body shapes. Validation using MRI-derived ground-truth models shows that our method creates participant-specific musculoskeletal models that are closer to the MRI-derived ground truth than scaled generic models. Additionally, we performed lab-based gait experiments to evaluate the effect of our method on residual forces and joint moments, where we found that our method leads to a reduction of residual forces of up to 14.9 % and a reduction of metabolic cost of up to 12.8 % when compared to generic musculoskeletal models. Our new generic models show similar joint moment outcomes, but less reduction of residual forces than the personalized models.

Realistic full-body avatars play a key role in representing users in virtual environments, where they have been shown to considerably improve important effects of immersive experiences such as body ownership and presence. Consequently, the demand for realistic virtual humans – and methods for creating them – is rapidly growing. However, despite extensive research into 3D reconstruction of avatars from real humans, an easy and affordable method for generating realistic and VR-capable avatars is still lacking: Existing methods are either limited to complex capture hardware and/or controlled lab environments, do not provide sufficient visual fidelity, or cannot be rendered at sufficient frame rates for multi-avatar VR applications. To make avatar reconstruction widely available, we developed Avatars for the Masses – a client-server-based online service for scanning real humans with an easy-to-use smartphone application that empowers even non-expert users to capture photorealistic and VR-ready avatars. The data captured by the smartphone is transferred to a reconstruction server, where the avatar is generated in a fully automated process. Our advancements in capturing and reconstructing allow for higher-quality avatars even in less controlled in-the-wild environments. Extensive qualitative and quantitative evaluations show our method’s avatars to be on par with the ones generated by expensive expert-operated systems. It also generates more accurate replicas in comparison to the current state of the art in smartphone-based reconstruction, produces much less artifacts and provides a much higher rendering performance in VR in comparison to three representative neural methods. A comprehensive user study confirms similar perception results compared to avatars reconstructed with expensive expert-operated systems, and it underscores a sufficient usability of the overall system. To truly bring avatars to the masses, we will make our smartphone application publicly available for research purposes. More details can be found on the project page: https://avatars.cs.tu-dortmund.de.

Due to the increasing use of virtual avatars, the animation of head‐hand interactions has recently gained attention. To this end, we present a novel volumetric and physics‐based interaction simulation. In contrast to previous work, our simulation incorporates temporal effects such as collision paths, respects anatomical constraints, and can detect and simulate skin pulling. As a result, we can achieve more natural‐looking interaction animations and take a step towards greater realism. However, like most complex and computationally expensive simulations, ours is not real‐time capable even on high‐end machines. Therefore, we train small and efficient neural networks as accurate approximations that achieve about 200 FPS on consumer GPUs, about 50 FPS on CPUs, and are learned in less than four hours for one person. In general, our focus is not to generalize the approximation networks to low‐resolution head models but to adapt them to more detailed personalized avatars. Nevertheless, we show that these networks can learn to approximate our head‐hand interaction model for multiple identities while maintaining computational efficiency.Since the quality of the simulations can only be judged subjectively, we conducted a comprehensive user study which confirms the improved realism of our approach. In addition, we provide extensive visual results and inspect the neural approximations quantitatively. All data used in this work has been recorded with a multi–view camera rig. Code and data are available at https://gitlab.cs.hs‐rm.de/cvmr_releases/HeadHand.

This paper investigates if and how self-similarity and having motor control impact sense of embodiment, self-identification, and body weight perception in Augmented Reality (AR). We conducted a 2x2 mixed design experiment involving 60 participants who interacted with either synchronously moving virtual humans or independently moving ones, each with self-similar or generic appearances, across two consecutive AR sessions. Participants evaluated their sense of embodiment, self-identification, and body weight perception of the virtual human. Our results show that self-similarity significantly enhanced sense of embodiment, self-identification, and the accuracy of body weight estimates with the virtual human. However, the effects of having motor control over the virtual human movements were notably weaker in these measures than in similar VR studies. Further analysis indicated that not only the virtual human itself but also the participants' body weight, self-esteem, and body shape concerns predict body weight estimates across all conditions. Our work advances the understanding of virtual human body weight perception in AR systems, emphasizing the importance of factors such as coherence with the real-world environment.

Working with scatterplots is a classic everyday task for data analysts, which gets increasingly complex the more plots are required to form an understanding of the underlying data. To help analysts retrieve relevant plots more quickly when they are needed, immersive virtual environments (iVEs) provide them with the option to freely arrange scatterplots in the 3D space around them. In this paper, we investigate the impact of different virtual environments on the users' ability to quickly find and retrieve individual scatterplots from a larger collection. We tested three different scenarios, all having in common that users were able to position the plots freely in space according to their own needs, but each providing them with varying numbers of landmarks serving as visual cues: an Empty scene as a baseline condition, a single landmark condition with one prominent visual cue being a Desk, and a multiple landmarks condition being a virtual Office. Results from a between-subject investigation with 45 participants indicate that the time and effort users invest in arranging their plots within an iVE had a greater impact on memory performance than the design of the iVE itself. We report on the individual arrangement strategies that participants used to solve the task effectively and underline the importance of an active arrangement phase for supporting the spatial memorization of scatterplots in iVEs.