Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition
Open Access
- 26 July 2023
- journal article
- research article
- Published by MDPI AG in Mathematics
- Vol. 11 (15) , 3280
Abstract
Previous research on 3D skeleton-based human action recognition has frequently relied on a sequence-wise viewpoint normalization process, which adjusts the view directions of all segmented action sequences. This type of approach typically demonstrates robustness against variations in viewpoint found in short-term videos, a characteristic commonly encountered in public datasets. However, our preliminary investigation of complex action sequences, such as discussions or smoking, reveals its limitations in capturing the intricacies of such actions. To address these view-dependency issues, we propose a straightforward, yet effective, sequence-wise augmentation technique. This strategy enhances the robustness of action recognition models, particularly against changes in viewing direction that mainly occur within the horizontal plane (azimuth) by rotating human key points around either the z-axis or the spine vector, effectively creating variations in viewing directions. We scrutinize the robustness of this approach against real-world viewpoint variations through extensive empirical studies on multiple public datasets, including an additional set of custom action sequences. Despite the simplicity of our approach, our experimental results consistently yield improved action recognition accuracies. Compared to the sequence-wise viewpoint normalization method used with advanced deep learning models like Conv1D, LSTM, and Transformer, our approach showed a relative increase in accuracy of 34.42% for the z-axis and 10.86% for the spine vector.Keywords
Funding Information
- Korea Evaluation Institute of Industrial Technology (20015188)
This publication has 30 references indexed in Scilit:
- Enhanced skeleton visualization for view invariant human action recognitionPattern Recognition, 2017
- NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis2016
- Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group2014
- Recognizing Actions from Depth Cameras as Weakly Aligned Multi-part Bag-of-Poses2013
- Real-time human pose recognition in parts from single depth imagesCommunications of the ACM, 2013
- View invariant human action recognition using histograms of 3D joints2012
- Microsoft Kinect Sensor and Its EffectIEEE MultiMedia, 2012
- Latent structured models for human pose estimation2011
- Cross-View Action Recognition from Temporal Self-similarities2008
- Long Short-Term MemoryNeural Computation, 1997