RadarSense: Accurate Recognition of Mid-air Hand Gestures with Radar Sensing and Few Training Examples
Open Access
- 11 September 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Interactive Intelligent Systems
- Vol. 13 (3) , 1-45
Abstract
Microwave radars bring many benefits to mid-air gesture sensing due to their large field of view and independence from environmental conditions, such as ambient light and occlusion. However, radar signals are highly dimensional and usually require complex deep learning approaches. To understand this landscape, we report results from a systematic literature review of ( N =118) scientific papers on radar sensing, unveiling a large variety of radar technology of different operating frequencies and bandwidths and antenna configurations but also various gesture recognition techniques. Although highly accurate, these techniques require a large amount of training data that depend on the type of radar. Therefore, the training results cannot be easily transferred to other radars. To address this aspect, we introduce a new gesture recognition pipeline that implements advanced full-wave electromagnetic modeling and inversion to retrieve physical characteristics of gestures that are radar independent, i.e., independent of the source, antennas, and radar-hand interactions. Inversion of radar signals further reduces the size of the dataset by several orders of magnitude, while preserving the essential information. This approach is compatible with conventional gesture recognizers, such as those based on template matching, which only need a few training examples to deliver high recognition accuracy rates. To evaluate our gesture recognition pipeline, we conducted user-dependent and user-independent evaluations on a dataset of 16 gesture types collected with the Walabot, a low-cost off-the-shelf array radar. We contrast these results with those obtained for the same gesture types collected with an ultra-wideband radar made of a vector network analyzer with a single horn antenna and with a computer vision sensor, respectively. Based on our findings, we suggest some design implications to support future development in radar-based gesture recognition.Keywords
Funding Information
- Wallonie-Bruxelles International (SUB/2021/519018 and UEFISCDI)
- RadarSense (PN-III-CEI-BIM-PBE-2020-0001/1BM/2021)
- Fonds de la Recherche Scientifique (40001931, 40011629)
This publication has 104 references indexed in Scilit:
- Analysis of the Accuracy and Robustness of the Leap Motion ControllerSensors, 2013
- Foundational Issues in Touch-Surface Stroke Gesture Design — An Integrative ReviewFoundations and Trends® in Human-Computer Interaction, 2012
- Reconstruction of sub-wavelength fractures and physical properties of masonry media using full-waveform inversion of proximal penetrating radarJournal of Applied Geophysics, 2011
- Systematic literature reviews in software engineering – A tertiary studyInformation and Software Technology, 2010
- Surface and borehole ground-penetrating-radar developmentsGeophysics, 2010
- uWave: Accelerometer-based personalized gesture recognition and its applicationsPervasive and Mobile Computing, 2009
- Human motion recognition using ultra-wideband radar and cameras on mobile robotTransactions of Tianjin University, 2009
- A Unifying Reference Framework for multi-target user interfacesInteracting with Computers, 2003
- Understanding and Using ContextPersonal and Ubiquitous Computing, 2001
- Tree visualization with tree-mapsACM Transactions on Graphics, 1992