NeuroAiR: Deep Learning Framework for Airwriting Recognition From Scalp-Recorded Neural Signals
- 25 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 73 (00189456) , 1-13
Abstract
Airwriting recognition is a task that involves identifying letters written in free space using finger movement. It is a special case of gesture recognition, where gestures correspond to letters in a specific language. Electroencephalography (EEG) is a non-invasive technique for recording brain activity and has been widely used in brain-computer interface applications. Leveraging EEG signals for airwriting recognition offers a promising alternative input method for Human-Computer Interaction. One key advantage of airwriting recognition is that users do not need to learn new gestures. By concatenating recognized letters, a wide range of words can be formed, making it applicable to a broader population. However, there has been limited research in the recognition of airwriting using EEG signals, which forms the core focus of this study. The NeuroAiR dataset comprising EEG signals recorded during writing English uppercase alphabets is first constructed. Various features are then explored in conjunction with different deep learning models to achieve accurate airwriting recognition. These features include processed EEG data, Independent Component Analysis components, source-domain-based scout time series, and spherical and head harmonic decomposition-based features. Furthermore, the impact of different EEG frequency bands on system performance is comprehensively investigated. The highest accuracy achieved in this study is 44.04% using Independent Component Analysis components and the EEGNet classification model. The results highlight the potential of EEG-based airwriting recognition as a user-friendly modality for alternative input methods in Human-Computer Interaction applications. This research sets a strong baseline for future advancements and demonstrates the viability and utility of EEG-based airwriting recognition.Keywords
All Related Versions
Funding Information
- Prime Minister’s Research Fellowship (PMRF), Ministry of Education (MoE), Government of India, under Grant PLN08/PMRF
- Infosys Foundation through the Young Investigator Award
This publication has 42 references indexed in Scilit:
- TripCEAiR: A multi-loss minimization approach for surface EMG based airwriting recognitionBiomedical Signal Processing and Control, 2023
- EEG based arm movement intention recognition towards enhanced safety in symbiotic Human-Robot CollaborationRobotics and Computer-Integrated Manufacturing, 2021
- High-performance brain-to-text communication via handwritingNature, 2021
- Fingertip detection and tracking for recognition of air-writing in videosExpert Systems with Applications, 2019
- Tuning characteristics of low-frequency EEG to positions and velocities in visuomotor and oculomotor tracking tasksScientific Reports, 2018
- Envisioned speech recognition using EEG sensorsPersonal and Ubiquitous Computing, 2017
- A review of hand gesture and sign language recognition techniquesInternational Journal of Machine Learning and Cybernetics, 2017
- Deep learning with convolutional neural networks for EEG decoding and visualizationHuman Brain Mapping, 2017
- Theory and Applications of Spherical Microphone Array Processing2017
- EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysisJournal of Neuroscience Methods, 2004