Fast and Expressive Gesture Recognition using a Combination-Homomorphic Electromyogram Encoder
Preprint
Abstract
We study the task of gesture recognition from electromyography (EMG), with the goal of enabling expressive human-computer interaction at high accuracy, while minimizing the time required for new subjects to provide calibration data. To fulfill these goals, we define combination gestures consisting of a direction component and a modifier component. New subjects only demonstrate the single component gestures and we seek to extrapolate from these to all possible single or combination gestures. We extrapolate to unseen combination gestures by combining the feature vectors of real single gestures to produce synthetic training data. This strategy allows us to provide a large and flexible gesture vocabulary, while not requiring new subjects to demonstrate combinatorially many example gestures. We pre-train an encoder and a combination operator using self-supervision, so that we can produce useful synthetic training data for unseen test subjects. To evaluate the proposed method, we collect a real-world EMG dataset, and measure the effect of augmented supervision against two baselines: a partially-supervised model trained with only single gesture data from the unseen subject, and a fully-supervised model trained with real single and real combination gesture data from the unseen subject. We find that the proposed method provides a dramatic improvement over the partially-supervised model, and achieves a useful classification accuracy that in some cases approaches the performance of the fully-supervised model.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- User Training With Error Augmentation for sEMG-Based Gesture ClassificationIEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024
- Transfer Learning on Electromyography (EMG) Tasks: Approaches and BeyondIEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023
- Comparing subject-to-subject transfer learning methods in surface electromyogram-based motion recognition with shallow and deep classifiersNeurocomputing, 2022
- A review on transfer learning in EEG signal analysisNeurocomputing, 2021
- Cross-Domain MLP and CNN Transfer Learning for Biological Signal Processing: EEG and EMGIEEE Access, 2020
- Design and Myoelectric Control of an Anthropomorphic Prosthetic HandJournal of Bionic Engineering, 2017
- Myoelectric control of prosthetic hands: state-of-the-art reviewMedical Devices: Evidence and Research, 2016
- Quantification and solutions of arm movements effect on sEMG pattern recognitionBiomedical Signal Processing and Control, 2014
- Classification of Simultaneous Movements Using Surface EMG Pattern RecognitionIEEE Transactions on Biomedical Engineering, 2012
- Feature reduction and selection for EMG signal classificationExpert Systems with Applications, 2012