Wearable Saudi Sign Language Recognition Device Based on Neural Network
Authors:
- 22 April 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 199-208
Abstract
In Saudi Arabia, over 750,000 individuals are deaf or mute, often facing communication barriers due to limited sign language familiarity among the general population. This research introduces a wearable device leveraging neural networks to recog-nize Saudi Sign Language, aiming to bridge this communication gap. The device simultaneously processes data from both hands, integrating inputs from the non-dominant hand with those from the dominant. Three primary phases define the system: data acquisition, machine learning model development, and real-time sign recognition. Data were gathered using electromyography (EMG) and inertial measurement unit (IMU) sensors, facilitated by the Arduino IDE, with two Arduino Nano 33 BLE micro-controllers managing collection and transmission. The dataset, comprising 25 signs from four subjects with each sign repeated 20 times, informed the machine learning process. Utilizing Python, a model incorporating dual Convolutional Neural Networks for feature extraction from sensor readings and dense neural networks for training was built. Subsequent sign recognition was realized by deploying the model on Arduino via TensorFlow Lite. Impressively, this innovative system boasts a 92 % accuracy rate, marking a significant stride toward inclusive communication solutions.Keywords
This publication has 13 references indexed in Scilit:
- Video Sign Language Recognition using Pose Extraction and Deep Learning Models2023
- Implementation of wearable glove for sign language expression based on deep learningMicrosystem Technologies, 2023
- Argument Structure and Word Order in Saudi Sign LanguageJournal of Language Teaching and Research, 2023
- Computer Vision-Based Bengali Sign Language To Text Generation2022
- Sign Language Recognition for Arabic Alphabets Using Transfer Learning TechniqueComputational Intelligence and Neuroscience, 2022
- Thai Sign Language Recognition: an Application of Deep Neural Network2021
- Real-Time Translation of Indian Sign Language using LSTM2019
- The Language of Glove: Wireless gesture decoder with low-power and stretchable hybrid electronicsPLOS ONE, 2017
- A Framework for Hand Gesture Recognition Based on Accelerometer and EMG SensorsIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2011
- Sign LanguageAnnual Review of Applied Linguistics, 1980