Learning to Recognize Human Actions From Noisy Skeleton Data Via Noise Adaptation
- 15 October 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 24 (15209210) , 1152-1163
Abstract
Recent studies have made great progress on skeleton-based action recognition. However, most of them are developed with relatively clean skeletons without the presence of intensive noise. We argue that the models learned from relatively clean data are not well generalizable to handle noisy skeletons commonly appeared in the real world. In this paper, we address the challenge of recognizing human actions from noisy skeletons, which is seldom explored by previous methods. Beyond exploring the new problem, we further take a new perspective to address it, i.e. , noise adaptation, which gets rid of explicit skeleton noise modeling and reliance on skeleton ground truths. Specifically, we develop regression-based and generation-based adaptation models according to whether pairs of noisy skeletons are available. The regression-based model aims to learn noise-suppressed intrinsic feature representations by mapping pairs of noisy skeletons into a noise-robust space. When only unpaired skeletons are accessible, the generation-based model aims to adapt the features from noisy skeletons to a low-noise space by adversarial learning. To verify our proposed model and facilitate research on noisy skeletons, we collect a new dataset Noisy Skeleton Dataset (NSD), the skeletons of which are with much noise and more similar to daily-life data than previous datasets. Extensive experiments are conducted on the NSD, VV-RGBD and N-UCLA datasets, and results consistently show the outstanding performance of our proposed model.Keywords
Funding Information
- National Key Research and Development Program of China (2018AAA0102702)
- Fundamental Research Funds for the Central Universities
- National Natural Science Foundation of China (61772043)
This publication has 41 references indexed in Scilit:
- Effective Active Skeleton Representation for Low Latency Human Action RecognitionIEEE Transactions on Multimedia, 2015
- Jointly learning heterogeneous features for RGB-D activity recognition2015
- Hierarchical recurrent neural network for skeleton based action recognition2015
- Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group2014
- Cross-View Action Modeling, Learning, and Recognition2014
- Mining actionlet ensemble for action recognition with depth cameras2012
- View invariant human action recognition using histograms of 3D joints2012
- Two-person interaction detection using body-pose features and multiple instance learning2012
- A survey of vision-based methods for action representation, segmentation and recognitionComputer Vision and Image Understanding, 2011
- Visual perception of biological motion and a model for its analysisPerception & Psychophysics, 1973