A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities
Top Cited Papers
Open Access
- 9 September 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 142606-142639
Abstract
Researchers have been developing Hand Gesture Recognition (HGR) systems to enhance natural, efficient, and authentic human-computer interaction, especially benefiting those who rely solely on hand gestures for communication. Despite significant progress, automatic and precise identification of hand gestures remains a considerable challenge in computer vision. Recent studies have focused on specific modalities like RGB images, skeleton data, and spatiotemporal interest points. This paper comprehensively reviews HGR techniques and data modalities from 2014 to 2024, exploring advancements in sensor technology and computer vision. We highlight accomplishments using various modalities, including RGB, Skeleton, Depth, Audio, Electromyography (EMG), Electroencephalography (EEG), and Multimodal approaches and identify areas needing further research. We reviewed over 200 articles from prominent databases, focusing on data collection, data settings, and gesture representation. Our review assesses the efficacy of HGR systems through their recognition accuracy and identifies a gap in research on continuous gesture recognition, indicating the need for improved vision-based gesture systems. The field has experienced steady research progress, including advancements in hand-crafted features and deep learning (DL) techniques. Additionally, we report on the promising developments in HGR methods and the area of multimodal approaches. We hope this survey will serve as a potential guideline for diverse data modality-based HGR research.Keywords
All Related Versions
Funding Information
- Japan Society for the Promotion of Science (JSPS) Grants-in-Aid for Scientific Research (JP23H03477)
This publication has 245 references indexed in Scilit:
- Literature review of vision‐based dynamic gesture recognition using deep learning techniquesConcurrency and Computation: Practice and Experience, 2022
- LSA-T: The First Continuous Argentinian Sign Language Dataset for Sign Language Translation2022
- Event-Related Potential Classification Based on EEG Data Using xDWAN with MDM and KNN2021
- Fully Convolutional Networks for Continuous Sign Language Recognition2020
- Stochastic Fine-Grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition2020
- The Korean Sign Language Dataset for Action Recognition2019
- Spatial-Temporal Graph Convolutional Networks for Sign Language Recognition2019
- A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion2019
- Hidden Two-Stream Convolutional Networks for Action Recognition2019
- Spatial-Temporal Attention Res-TCN for Skeleton-Based Dynamic Hand Gesture Recognition2019