Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer‐Grade VR Devices
Open Access
- 1 December 2022
- journal article
- conference paper
- Published by Wiley in Computer Graphics Forum
- Vol. 41 (8) , 107-118
Abstract
The animation of user avatars plays a crucial role in conveying their pose, gestures, and relative distances to virtual objects or other users. Self‐avatar animation in immersive VR helps improve the user experience and provides a Sense of Embodiment. However, consumer‐grade VR devices typically include at most three trackers, one at the Head Mounted Display (HMD), and two at the handheld VR controllers. Since the problem of reconstructing the user pose from such sparse data is ill‐defined, especially for the lower body, the approach adopted by most VR games consists of assuming the body orientation matches that of the HMD, and applying animation blending and time‐warping from a reduced set of animations. Unfortunately, this approach produces noticeable mismatches between user and avatar movements. In this work we present a new approach to animate user avatars that is suitable for current mainstream VR devices. First, we use a neural network to estimate the user's body orientation based on the tracking information from the HMD and the hand controllers. Then we use this orientation together with the velocity and rotation of the HMD to build a feature vector that feeds a Motion Matching algorithm. We built a MoCap database with animations of VR users wearing a HMD and used it to test our approach on both self‐avatars and other users' avatars. Our results show that our system can provide a large variety of lower body animations while correctly matching the user orientation, which in turn allows us to represent not only forward movements but also stepping in any direction.Keywords
All Related Versions
Funding Information
- Ministerio de Ciencia e Innovación (PID2021‐122136OB‐C21)
This publication has 27 references indexed in Scilit:
- A Survey of Full-Body Motion Reconstruction in Immersive Virtual Reality ApplicationsIEEE Transactions on Visualization and Computer Graphics, 2019
- Improving Walking in Place Methods with Individualization and Deep Networks2019
- Investigation of Visual Self-Representation for a Walking-in-Place Navigation System in Virtual Reality2019
- Mo2Cap2
: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye CameraIEEE Transactions on Visualization and Computer Graphics, 2019
- Deep inertial poserACM Transactions on Graphics, 2018
- Users' locomotor behavior in collaborative virtual reality2018
- High performance animation in Gears of War 42017
- Real-time step detection using the integrated sensors of a head-mounted display2017
- Generalizeability of Latency Detection in a Variety of Virtual EnvironmentsProceedings of the Human Factors and Ergonomics Society Annual Meeting, 2004
- Taking stepsACM Transactions on Computer-Human Interaction, 1995