Human action recognition in immersive virtual reality based on multi‐scale spatio‐temporal attention network
Top Cited Papers
- 23 September 2024
- journal article
- research article
- Published by Wiley in Computer Animation and Virtual Worlds
- Vol. 35 (5)
Abstract
Wearable human action recognition (HAR) has practical applications in daily life. However, traditional HAR methods solely focus on identifying user movements, lacking interactivity and user engagement. This paper proposes a novel immersive HAR method called MovPosVR. Virtual reality (VR) technology is employed to create realistic scenes and enhance the user experience. To improve the accuracy of user action recognition in immersive HAR, a multi‐scale spatio‐temporal attention network (MSSTANet) is proposed. The network combines the convolutional residual squeeze and excitation (CRSE) module with the multi‐branch convolution and long short‐term memory (MCLSTM) module to extract spatio‐temporal features and automatically select relevant features from action signals. Additionally, a multi‐head attention with shared linear mechanism (MHASLM) module is designed to facilitate information interaction, further enhancing feature extraction and improving accuracy. The MSSTANet network achieves superior performance, with accuracy rates of 99.33% and 98.83% on the publicly available WISDM and PAMPA2 datasets, respectively, surpassing state‐of‐the‐art networks. Our method showcases the potential to display user actions and position information in a virtual world, enriching user experiences and interactions across diverse application scenarios.Keywords
Funding Information
- China Scholarship Council (202208420109)
- National Natural Science Foundation of China (62202346)
This publication has 46 references indexed in Scilit:
- A deep local-temporal architecture with attention for lightweight human activity recognitionApplied Soft Computing, 2023
- Low-thrust station-keeping towards exploiting the inclined geosynchronous dynamicsAdvances In Space Research, 2023
- Transformer-based deep reverse attention network for multi-sensory human activity recognitionEngineering Applications of Artificial Intelligence, 2023
- Key technologies of human–computer interaction for immersive somatosensory interactive games using VR technologySoft Computing, 2022
- DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensorsApplied Soft Computing, 2021
- Attention induced multi-head convolutional neural network for human activity recognitionApplied Soft Computing, 2021
- Human posture tracking with flexible sensors for motion recognitionComputer Animation and Virtual Worlds, 2021
- A Gaussian mixture based hidden Markov model for motion recognition with 3D vision deviceComputers and Electrical Engineering, 2020
- Motion Rehab AVE 3D: A VR-based exergame for post-stroke rehabilitationComputer Methods and Programs in Biomedicine, 2017
- Effect of Interaction Based on Augmented Context in Immersive Virtual Reality EnvironmentWireless Personal Communications, 2017