Expanding Indian Sign Language Recognition System using Class Incremental Learning
Author:
- 10 November 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 6, 1-5
Abstract
Sign language provides a visual means of communication for hearing impaired people through hand gestures, movements and facial expressions. In literature, several Sign Language Recognition (SLR) systems have been developed using deep learning algorithms. However, the performance of the SLR systems have been reported for a finite number of signs. In this work, class incremental learning is proposed to enable convolutional neural network (CNN) for recognizing new set of signs, given its knowledge about previously available set of signs. The network is trained in steps by introducing a few new classes in each step. The network is trained on a given set of classes and then retrained on training data consist of labelled samples of new classes and a few exemplars of previously learned classes. Experimental results are presented for incremental learning of 100 sign gestures from the self-recorded multi-sensor dataset of Indian Sign Language (ISL). The proposed model could classify new signs with a best average accuracy of 92.6%, while avoiding catastrophic forgetting and maintaining an accuracy of 80.81% for previously learned set of signs.Keywords
This publication has 8 references indexed in Scilit:
- Class-Incremental Learning: Survey and Performance Evaluation on Image ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- Essentials for Class Incremental Learning2021
- Indian sign language recognition using wearable sensors and multi-label classificationComputers and Electrical Engineering, 2021
- Mnemonics Training: Multi-Class Incremental Learning Without Forgetting2020
- Learning a Unified Classifier Incrementally via Rebalancing2019
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017
- Convolutional neural networks for time series classificationJournal of Systems Engineering and Electronics, 2017
- Vision-Based Portuguese Sign Language Recognition System2014