Multimodal Learning for Human Action Recognition Via Bimodal/Multimodal Hybrid Centroid Canonical Correlation Analysis
- 11 October 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 21 (5) , 1317-1331
Abstract
In this paper, we study the problem of human action recognition from multiple feature modalities. We propose Bimodal Hybrid Centroid Canonical Correlation Analysis (BHCCCA) and Multimodal Hybrid Centroid Canonical Correlation Analysis (MHCCCA) to learn the discriminative and informative shared space, by considering the correlation among different classes across two modalities (BHCCCA) and three or more modalities (MHCCCA). We then introduce a new human action recognition framework by using BHCCCA/MHCCCA for fusing different modalities (RGB, depth, skeleton, and accelerometer data). Performance evaluation on four publicly accessible datasets (MSR Action3D, UTD-MHAD, UTD-MHAD-Kinect V2, and Berkeley MHAD) demonstrated the effectiveness of the proposed framework.Keywords
This publication has 46 references indexed in Scilit:
- Robust 3D Action Recognition with Random Occupancy Patterns2012
- STOP: Space-Time Occupancy Patterns for 3D Action Recognition from Depth Map Sequences2012
- Transfer Discriminant-Analysis of Canonical Correlations for View-Transfer Action Recognition2012
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Intelligent video surveillance system: 3-tier context-aware surveillance system with metadataMultimedia Tools and Applications, 2010
- Local Descriptors for Spatio-temporal Recognition2006
- On Space-Time Interest PointsInternational Journal of Computer Vision, 2005
- Canonical Correlation Analysis: An Overview with Application to Learning MethodsNeural Computation, 2004
- The recognition of human movement using temporal templatesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Visual Motion PerceptionScientific American, 1975