Improving Hand Gesture Recognition Robustness to Dynamic Posture Variations by Multimodal Deep Feature Fusion
Open Access
- 22 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 32 (15344320) , 3104-3115
Abstract
Surface electromyography (sEMG), a human-machine interface for gesture recognition, has shown promising potential for decoding motor intentions, but a variety of nonideal factors restrict its practical application in assistive robots. In this paper, we summarized the current mainstream gesture recognition strategies and proposed a gesture recognition method based onmultimodal canonical correlation analysis feature fusion classification (MCAFC) for a nonideal condition that occurs in daily life, i.e., posture variations. The deep features of the sEMG and acceleration signals were first extracted via convolutional neural networks. A canonical correlation analysis was subsequently performed to associate the deep features of the two modalities. The transformed features were utilized as inputs to a linear discriminant analysis classifier to recognize the corresponding gestures. Both offline and real-time experiments were conducted on eight able-bodied subjects. The experimental results indicated that MCAFC achieved an average classification accuracy, average motion completion rate, and average motion completion time of 93.44%, 94.05%, and 1.38 s, respectively, with multiple dynamic postures, indicating significantly better performance than that of comparable methods. The results demonstrate the feasibility and superiority of the proposed multimodal signal feature fusion method for gesture recognition with posture variations, providing a new scheme for myoelectric control.Keywords
Funding Information
- National Natural Science Foundation of China (U22A2067, 62103406, 62333007)
- Applied Basic Research Program Project of Liaoning Province, China, (2022JH2/101300102)
This publication has 43 references indexed in Scilit:
- Long-term upper-extremity prosthetic control using regenerative peripheral nerve interfaces and implanted EMG electrodesJournal of Neural Engineering, 2023
- Persian sign language recognition using IMU and surface EMG sensorsMeasurement, 2021
- Hilbert sEMG data scanning for hand gesture recognition based on deep learningNeural Computing and Applications, 2020
- Gesture recognition by instantaneous surface EMG imagesScientific Reports, 2016
- Electromyography data for non-invasive naturally-controlled robotic hand prosthesesScientific Data, 2014
- Quantification and solutions of arm movements effect on sEMG pattern recognitionBiomedical Signal Processing and Control, 2014
- Feature reduction and selection for EMG signal classificationExpert Systems with Applications, 2012
- Improving myoelectric pattern recognition positional robustness using advanced training protocols2011
- Targeted Muscle Reinnervation for Real-time Myoelectric Control of Multifunction Artificial ArmsJAMA, 2009
- A new strategy for multifunction myoelectric controlIEEE Transactions on Biomedical Engineering, 1993