Real-Time Bangla Sign Language Detection with Sentence and Speech Generation
Authors:
- 19 December 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
Sign language, the non-verbal language used by the people with hearing and speaking disability, known as the deaf and mute, to connect the bridge of communication with others. Being a visual means of communication, it deprives the mutes to communicate with people having a visual impairment. A medium recognizing sign language and converting it into text and speech could fill the gap. Recently lots of research has been on Bangla Sign Language(BdSL) classification, not on Bangla sentence and speech generation. Most of them classify either digits or alphabets and also face time delay. This paper proposes a system for BdSL recognition that can interpret BdSL from a sequence of images or a video stream and generate both textual sentences and speech in real-time. We have used YOLOv4 as the object detection model. We have also proposed three new signs for the sentence generation task and built a dataset consisting of 12.5k BdSL images of 49 different classes where 39 are Bangla alphabets, 10 are Bangla digits, and the three new proposed signs.Keywords
This publication has 16 references indexed in Scilit:
- Thai finger spelling localization and classification under complex background using a YOLO-based deep learning2019
- Ishara-Lipi: The First Complete MultipurposeOpen Access Dataset of Isolated Characters for Bangla Sign Language2018
- Bangla Sign Language Detection Using SIFT and CNN2018
- Path Aggregation Network for Instance Segmentation2018
- Hand sign language recognition for Bangla alphabet based on Freeman Chain Code and ANN2017
- Feature Pyramid Networks for Object Detection2017
- Hand sign language recognition for Bangla alphabet using Support Vector Machine2016
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Real-time computer vision-based Bengali Sign Language recognition2014
- The effects of modality: linguistic universals and sign language universals2006