Abstract
Sign language provides a visual means of communication for hearing impaired people through hand gestures, movements and facial expressions. In literature, several Sign Language Recognition (SLR) systems have been developed using deep learning algorithms. However, the performance of the SLR systems have been reported for a finite number of signs. In this work, class incremental learning is proposed to enable convolutional neural network (CNN) for recognizing new set of signs, given its knowledge about previously available set of signs. The network is trained in steps by introducing a few new classes in each step. The network is trained on a given set of classes and then retrained on training data consist of labelled samples of new classes and a few exemplars of previously learned classes. Experimental results are presented for incremental learning of 100 sign gestures from the self-recorded multi-sensor dataset of Indian Sign Language (ISL). The proposed model could classify new signs with a best average accuracy of 92.6%, while avoiding catastrophic forgetting and maintaining an accuracy of 80.81% for previously learned set of signs.

This publication has 8 references indexed in Scilit: