On the Effect of Training Convolution Neural Network for Millimeter-Wave Radar-Based Hand Gesture Recognition
Open Access
Abstract
The purpose of this paper was to investigate the effect of a training state-of-the-art convolution neural network (CNN) for millimeter-wave radar-based hand gesture recognition (MR-HGR). Focusing on the small training dataset problem in MR-HGR, this paper first proposed to transfer the knowledge with the CNN models in computer vision to MR-HGR by fine-tuning the models with radar data samples. Meanwhile, for the different data modality in MR-HGR, a parameterized representation of temporal space-velocity (TSV) spectrogram was proposed as an integrated data modality of the time-evolving hand gesture features in the radar echo signals. The TSV spectrograms representing six common gestures in human–computer interaction (HCI) from nine volunteers were used as the data samples in the experiment. The evaluated models included ResNet with 50, 101, and 152 layers, DenseNet with 121, 161 and 169 layers, as well as light-weight MobileNet V2 and ShuffleNet V2, mostly proposed by many latest publications. In the experiment, not only self-testing (ST), but also more persuasive cross-testing (CT), were implemented to evaluate whether the fine-tuned models generalize to the radar data samples. The CT results show that the best fine-tuned models can reach to an average accuracy higher than 93% with a comparable ST average accuracy almost 100%. Moreover, in order to alleviate the problem caused by private gesture habits, an auxiliary test was performed by augmenting four shots of the gestures with the heaviest misclassifications into the training set. This enriching test is similar with the scenario that a tablet reacts to a new user. The results of two different volunteer in the enriching test shows that the average accuracy of the enriched gesture can be improved from 55.59% and 65.58% to 90.66% and 95.95% respectively. Compared with some baseline work in MR-HGR, the investigation by this paper can be beneficial in promoting MR-HGR in future industry applications and consumer electronic design.Keywords
Funding Information
- The China National Natural Science Foundation Project (61601141 and 620171152)
This publication has 20 references indexed in Scilit:
- Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar SensorIEEE Sensors Journal, 2018
- Radar-based hand gesture recognition using I-Q echo plot and convolutional neural network2017
- Densely Connected Convolutional Networks2017
- Xception: Deep Learning with Depthwise Separable Convolutions2017
- Interacting with Soli2016
- Hand Gesture Recognition Using Micro-Doppler Signatures With Convolutional Neural NetworkIEEE Access, 2016
- SoliACM Transactions on Graphics, 2016
- Deep Residual Learning for Image Recognition2016
- Hand Body Language Gesture Recognition Based on Signals From Specialized Glove and Machine Learning AlgorithmsIEEE Transactions on Industrial Informatics, 2016
- A Framework for Hand Gesture Recognition Based on Accelerometer and EMG SensorsIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2011