ST-Gesture: Spatio-Temporal Feature Extraction Network for Gesture Recognition Based on Attention Mechanism

Authors:

Abstract
Gesture recognition, a crucial technology in human-computer interaction, finds applications across various domains, including smart homes, automotive driver assistance systems, and more. However, existing millimeter-wave radar-based gesture recognition algorithms often face challenges in achieving high accuracy and robustness in complex environments. To address these limitations, this paper proposes a Space-Time Gesture Recognition algorithm (ST-Gesture) that leverages attention mechanisms. The approach first exploits the multi-dimensional information within mmWave radar data cubes. ST-Gesture processes the Range-Azimuth-Doppler (RAD) data to generate three types of spectrograms: Range-Azimuth Map (RAM), Azimuth-Doppler Map (ADM), and Range-Doppler Map (RDM). Spatial and temporal features are then extracted from these spectrograms using an enhanced model that combines Convolutional Neural Networks (CNNs) with Temporal Convolutional Networks (TCNs). A cross-attention mechanism is employed to fuse complementary information from the different spectrograms, thereby improving recognition accuracy. Experimental results, based on both a self-constructed dataset encompassing diverse environments and positions, as well as public dataset, demonstrate that ST-Gesture outperforms state-of-the-art methods, achieving up to 99.2% accuracy. Additionally, the system's robustness to environmental and positional variations is thoroughly validated.

This publication has 45 references indexed in Scilit: