Deep Lip Reading-A Deep Learning Based Lip-Reading Software for the Hearing Impaired
- 1 November 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 40-44
Abstract
Lip reading is the task of decoding and understanding speech from the movement of a speaker's mouth. This can be extremely beneficial for aiding the hearing impaired to ‘listen’ to people who do not know sign language in real-world environments with a lot of noise pollution. Orthodoxically methods have focused mainly on heavy preprocessing. Despite showing tremendous potential, application of deep learning algorithms has been limited in this field. Here we present a convolutional neural network model to predict words from videos without any audio. It is developed using the pre-trained deep learning architecture VGG Net, pre-trained on the ImageNet Database with some custom modifications on the MIRACL-VCl Dataset of 10 words. The model achieved an accuracy of 94.86% in training, 93.82% in validation and 60% in testing. An app has been developed using this model which can use cloud computing to run the model real time in any smartphone to aid the hearing-impaired in their day-to-day activities and can make conversations with hearing impaired people more natural, organic as well as cost friendly.Keywords
This publication has 5 references indexed in Scilit:
- Dynamic Stream Weighting for Turbo-Decoding-Based Audiovisual ASR2016
- Unified System for Visual Speech Recognition and Speaker IdentificationLecture Notes in Computer Science, 2015
- A New Visual Speech Recognition Approach for RGB-D Cameras2014
- Lip reading using optical flow and support vector machines2010
- VISUAL SPEECH RECOGNITION USING DYNAMIC FEATURES AND SUPPORT VECTOR MACHINESInternational Journal of Image and Graphics, 2008