Human Action Recognition Based on Skeleton Information and Multi-Feature Fusion
Open Access
- 1 September 2023
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 12 (17) , 3702
Abstract
Action assessment and feedback can effectively assist fitness practitioners in improving exercise benefits. In this paper, we address key challenges in human action recognition and assessment by proposing innovative methods that enhance performance while reducing computational complexity. Firstly, we present Oct-MobileNet, a lightweight backbone network, to overcome the limitations of the traditional OpenPose algorithm’s VGG19 network, which exhibits a large parameter size and high device requirements. Oct-MobileNet employs octave convolution and attention mechanisms to improve the extraction of high-frequency features from the human body contour, resulting in enhanced accuracy with reduced model computational burden. Furthermore, we introduce a novel approach for action recognition that combines skeleton-based information and multiple feature fusion. By extracting spatial geometric and temporal characteristics from actions, we employ a sliding window algorithm to integrate these features. Experimental results show the effectiveness of our approach, demonstrating its ability to accurately recognize and classify various human actions. Additionally, we address the evaluation of traditional fitness exercises, specifically focusing on the BaDunJin movements. We propose a multimodal information-based assessment method that combines pose detection and keypoint analysis. Label sequences are obtained through a pose detector and each frame’s keypoint coordinates are represented as pose vectors. Leveraging multimodal information, including label sequences and pose vectors, we explore action similarity and perform quantitative evaluations to help exercisers assess the quality of their exercise performance.Keywords
This publication has 24 references indexed in Scilit:
- Cascaded Pyramid Network for Multi-person Pose Estimation2018
- A Review Study on the Beneficial Effects of BaduanjinThe Journal of Alternative and Complementary Medicine, 2018
- RMPE: Regional Multi-person Pose Estimation2017
- From action to activity: Sensor-based activity recognitionNeurocomputing, 2016
- MEMS inertial sensors: A tutorial overviewIEEE Communications Magazine, 2013
- Swimming Stroke Kinematic Analysis with BSN2010
- K-nearest neighborScholarpedia, 2009
- ZigBee wireless standardELMAR 2007, 2006
- Support vector machinesIEEE Intelligent Systems and their Applications, 1998
- Principal component analysisChemometrics and Intelligent Laboratory Systems, 1987