A union of deep learning and swarm-based optimization for 3D human action recognition
Open Access
- 31 March 2022
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 12 (1) , 1-17
Abstract
Human Action Recognition (HAR) is a popular area of research in computer vision due to its wide range of applications such as surveillance, health care, and gaming, etc. Action recognition based on 3D skeleton data allows simplistic, cost-efficient models to be formed making it a widely used method. In this work, we proposeDSwarm-Net, a framework that employs deep learning and swarm intelligence-based metaheuristic for HAR that uses 3D skeleton data for action classification. We extract four different types of features from the skeletal data namely: Distance, Distance Velocity, Angle, and Angle Velocity, which capture complementary information from the skeleton joints for encoding them into images. Encoding the skeleton data features into images is an alternative to the traditional video-processing approach and it helps in making the classification task less complex. The Distance and Distance Velocity encoded images have been stacked depth-wise and fed into a Convolutional Neural Network model which is a modified version of Inception-ResNet. Similarly, the Angle and Angle Velocity encoded images have been stacked depth-wise and fed into the same network. After training these models, deep features have been extracted from the pre-final layer of the networks, and the obtained feature representation is optimized by a nature-inspired metaheuristic, called Ant Lion Optimizer, to eliminate the non-informative or misleading features and to reduce the dimensionality of the feature set. DSwarm-Net has been evaluated on three publicly available HAR datasets, namely UTD-MHAD, HDM05, and NTU RGB+D 60 achieving competitive results, thus confirming the superiority of the proposed model compared to state-of-the-art models.Keywords
Funding Information
- National Agency for Academic Exchange of Poland (PPI/APM/2018/1/00004)
This publication has 76 references indexed in Scilit:
- Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigmKnowledge-Based Systems, 2015
- FaceNet: A unified embedding for face recognition and clustering2015
- The Ant Lion OptimizerAdvances in Engineering Software, 2015
- Statistical validation of multiple classifiers over multiple datasets in the field of pattern recognitionInternational Journal of Applied Pattern Recognition, 2015
- 3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian ManifoldIEEE Transactions on Cybernetics, 2014
- Multiple/Single-View Human Action Recognition via Part-Induced Multitask Structural LearningIEEE Transactions on Cybernetics, 2014
- Grey Wolf OptimizerAdvances in Engineering Software, 2014
- Spatio-Temporal Laplacian Pyramid Coding for Action RecognitionIEEE Transactions on Cybernetics, 2013
- A new feature selection algorithm based on binary ant colony optimization2013
- Mixed variable structural optimization using Firefly AlgorithmComputers & Structures, 2011