Depth Pooling Based Large-Scale 3-D Action Recognition With Convolutional Neural Networks
- 3 April 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 20 (5) , 1051-1061
Abstract
This paper proposes three simple, compact yet effective representations of depth sequences, referred to respectively as dynamic depth images (DDI), dynamic depth normal images (DDNI), and dynamic depth motion normal images (DDMNI), for both isolated and continuous action recognition. These dynamic images are constructed from a segmented sequence of depth maps using hierarchical bidirectional rank pooling to effectively capture the spatial-temporal information. Specifically, DDI exploits the dynamics of postures over time, and DDNI and DDMNI exploit the 3-D structural information captured by depth maps. Upon the proposed representations, a convolutional neural network (ConvNet)-based method is developed for action recognition. The image-based representations enable us to fine-tune the existing ConvNet models trained on image data without training a large number of parameters from scratch. The proposed method achieved the state-of-art results on three large datasets, namely, the large-scale continuous gesture recognition dataset (means the Jaccard index 0.4109), the large-scale isolated gesture recognition dataset (59.21%), and the NTU RGB+D dataset (87.08% cross-subject and 84.22% cross-view) even though only the depth modality was used.Keywords
All Related Versions
This publication has 47 references indexed in Scilit:
- Recent methods and databases in vision-based hand gesture recognition: A reviewComputer Vision and Image Understanding, 2015
- Survey on 3D Hand Gesture RecognitionIEEE Transactions on Circuits and Systems for Video Technology, 2015
- Caffe2014
- Human activity recognition from 3D data: A reviewPattern Recognition Letters, 2014
- CSMMI: Class-Specific Maximization of Mutual Information for Action and Gesture RecognitionIEEE Transactions on Image Processing, 2014
- Range-Sample Depth Feature for Action Recognition2014
- 3D SMoSIFT: three-dimensional sparse motion scale invariant feature transform for activity recognition from RGB-D videosJournal of Electronic Imaging, 2014
- The ChaLearn gesture dataset (CGD 2011)Machine Vision and Applications, 2014
- Action Recognition with Improved Trajectories2013
- HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences2013