Novel Wearable System to Recognize Sign Language in Real Time
Open Access
Abstract
The aim of this study is to develop a practical software solution for real-time recognition of sign language words using two arms. This will facilitate communication between hearing-impaired individuals and those who can hear. We are aware of several sign language recognition systems developed using different technologies, including cameras, armbands, and gloves. However, the system we propose in this study stands out for its practicality, utilizing surface electromyography (muscle activity) and inertial measurement unit (motion dynamics) data from both arms. We address the drawbacks of other methods, such as high costs, low accuracy due to ambient light and obstacles, and complex hardware requirements, which have limited their practical application. Our software can run on different operating systems using digital signal processing and machine learning methods specific to this study. For the test, we created a dataset of 80 words based on their frequency of use in daily life and performed a thorough feature extraction process. We tested the recognition performance using various classifiers and parameters and compared the results. The random forest algorithm emerged as the most successful, achieving a remarkable 99.875% accuracy, while the naïve Bayes algorithm had the lowest success rate with 87.625% accuracy. The new system promises to significantly improve communication for people with hearing disabilities and ensures seamless integration into daily life without compromising user comfort or lifestyle quality.Keywords
All Related Versions
This publication has 40 references indexed in Scilit:
- Spatial–temporal feature-based End-to-end Fourier network for 3D sign language recognitionExpert Systems with Applications, 2024
- IDF-Sign: Addressing Inconsistent Depth Features for Dynamic Sign Word RecognitionIEEE Access, 2023
- Multi-Semantic Discriminative Feature Learning for Sign Gesture Recognition Using Hybrid Deep Neural ArchitectureIEEE Access, 2023
- American Sign Language Words Recognition Using Spatio-Temporal Prosodic and Angle Features: A Sequential Learning ApproachIEEE Access, 2022
- ELM based two-handed dynamic Turkish Sign Language (TSL) word recognitionExpert Systems with Applications, 2021
- Recognition of Turkish Sign Language (TID) Using sEMG Sensor2020
- A review of hand gesture and sign language recognition techniquesInternational Journal of Machine Learning and Cybernetics, 2017
- PSGMiner: A modular software for polysomnographic analysisComputers in Biology and Medicine, 2016
- Real-time American sign language recognition using desk and wearable computer based videoIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- Glove-Talk: a neural network interface between a data-glove and a speech synthesizerIEEE Transactions on Neural Networks, 1993