ST-Gesture: Spatio-Temporal Feature Extraction Network for Gesture Recognition Based on Attention Mechanism
- 18 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Aerospace and Electronic Systems
- Vol. 62 (00189251) , 2188-2200
Abstract
Gesture recognition, a crucial technology in human-computer interaction, finds applications across various domains, including smart homes, automotive driver assistance systems, and more. However, existing millimeter-wave radar-based gesture recognition algorithms often face challenges in achieving high accuracy and robustness in complex environments. To address these limitations, this paper proposes a Space-Time Gesture Recognition algorithm (ST-Gesture) that leverages attention mechanisms. The approach first exploits the multi-dimensional information within mmWave radar data cubes. ST-Gesture processes the Range-Azimuth-Doppler (RAD) data to generate three types of spectrograms: Range-Azimuth Map (RAM), Azimuth-Doppler Map (ADM), and Range-Doppler Map (RDM). Spatial and temporal features are then extracted from these spectrograms using an enhanced model that combines Convolutional Neural Networks (CNNs) with Temporal Convolutional Networks (TCNs). A cross-attention mechanism is employed to fuse complementary information from the different spectrograms, thereby improving recognition accuracy. Experimental results, based on both a self-constructed dataset encompassing diverse environments and positions, as well as public dataset, demonstrate that ST-Gesture outperforms state-of-the-art methods, achieving up to 99.2% accuracy. Additionally, the system's robustness to environmental and positional variations is thoroughly validated.Keywords
This publication has 45 references indexed in Scilit:
- SML: A Skeleton-based multi-feature learning method for sign language recognitionKnowledge-Based Systems, 2024
- An effective framework of human abnormal behaviour recognition and tracking using multiscale dilated assisted residual attention networkExpert Systems with Applications, 2024
- A Multi-Modal Transformer network for action detectionPattern Recognition, 2023
- Global spatio-temporal synergistic topology learning for skeleton-based action recognitionPattern Recognition, 2023
- mmGesture: Semi-supervised gesture recognition system using mmWave radarExpert Systems with Applications, 2022
- Human action recognition using attention based LSTM network with dilated CNN featuresFuture Generation Computer Systems, 2021
- Recognizing activities of daily living from UWB radars and deep learningExpert Systems with Applications, 2021
- GigaHertz: Gesture Sensing Using Microwave Radar and IR Sensor with Machine Learning Algorithms2020
- Deep residual learning for image steganalysisMultimedia Tools and Applications, 2017
- Hand posture and gesture recognition techniques for virtual reality applications: a surveyVirtual Reality, 2016