Information Fusion for Human Action Recognition via Biset/Multiset Globality Locality Preserving Canonical Correlation Analysis
- 12 July 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 27 (11) , 5275-5287
Abstract
In this paper, we study the problem of human action recognition, in which each action is captured by multiple sensors and represented by multisets. We propose two novel information fusion techniques for fusing the information from multisets. The first technique is biset globality locality preserving canonical correlation analysis (BGLPCCA), which aims to learn the common feature subspace between two sets. The second technique is multiset globality locality preserving canonical correlation analysis (MGLPCCA), which aims to deal with three or more sets. The proposed BGLPCCA and MGLPCCA are able to learn a low-dimensional common subspace that preserves the local and global structures of data samples. Moreover, two novel descriptors are presented for both depth and skeleton. We then propose a new human action recognition framework employing the proposed BGLPCCA or MGLPCCA to learn the shared subspace from multiple sets of features including skeleton, depth, and optical flow. Extensive experiments on five publicly available datasets (MSR Action3D, UTD multimodal human action dataset, multimodal action database, Kinect activity recognition dataset, and SBU Kinect interaction dataset) demonstrate the effectiveness of the proposed framework.Keywords
This publication has 57 references indexed in Scilit:
- Evolutionary joint selection to improve human action recognition with RGB-D devicesExpert Systems with Applications, 2014
- Multiset Canonical Correlations Using Globality Preserving Projections With Applications to Feature Extraction and RecognitionIEEE Transactions on Neural Networks and Learning Systems, 2013
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Intelligent video surveillance system: 3-tier context-aware surveillance system with metadataMultimedia Tools and Applications, 2010
- Locality preserving CCA with applications to data visualization and pose estimationImage and Vision Computing, 2007
- On Space-Time Interest PointsInternational Journal of Computer Vision, 2005
- Canonical Correlation Analysis: An Overview with Application to Learning MethodsNeural Computation, 2004
- Feature fusion: parallel strategy vs. serial strategyPattern Recognition, 2003
- The recognition of human movement using temporal templatesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Visual Motion PerceptionScientific American, 1975