Deep Temporal-Spatial Feature Learning for Motor Imagery-Based Brain–Computer Interfaces
- 21 September 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 28 (11) , 2356-2366
Abstract
Motor imagery (MI) decoding is an important part of brain-computer interface (BCI) research, which translates the subject’s intentions into commands that external devices can execute. The traditional methods for discriminative feature extraction, such as common spatial pattern (CSP) and filter bank common spatial pattern (FBCSP), have only focused on the energy features of the electroencephalography (EEG) and thus ignored the further exploration of temporal information. However, the temporal information of spatially filtered EEG may be critical to the performance improvement of MI decoding. In this paper, we proposed a deep learning approach termed filter-bank spatial filtering and temporal-spatial convolutional neural network (FBSF-TSCNN) for MI decoding, where the FBSF block transforms the raw EEG signals into an appropriate intermediate EEG presentation, and then the TSCNN block decodes the intermediate EEG signals. Moreover, a novel stage-wise training strategy is proposed to mitigate the difficult optimization problem of the TSCNN block in the case of insufficient training samples. Firstly, the feature extraction layers are trained by optimization of the triplet loss. Then, the classification layers are trained by optimization of the cross-entropy loss. Finally, the entire network (TSCNN) is fine-tuned by the back-propagation (BP) algorithm. Experimental evaluations on the BCI IV 2a and SMR-BCI datasets reveal that the proposed stage-wise training strategy yields significant performance improvement compared with the conventional end-to-end training strategy, and the proposed approach is comparable with the state-of-the-art method.Keywords
Funding Information
- National Natural Science Foundation of China (61836003, 61573150, 61573152)
This publication has 33 references indexed in Scilit:
- Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2bFrontiers in Neuroscience, 2012
- Movement imagery-related lateralization of event-related (de)synchronization (ERD/ERS): Motor-imagery duration effectsClinical Neurophysiology, 2011
- A semi-supervised support vector machine approach for parameter setting in motor imagery-based brain computer interfacesCognitive Neurodynamics, 2010
- Cortical activity during motor execution, motor imagery, and imagery-based online feedbackProceedings of the National Academy of Sciences, 2010
- The Berlin Brain–Computer Interface: Non-Medical Uses of BCI TechnologyFrontiers in Neuroscience, 2010
- Cognitive motor processes: The role of motor imagery in the study of motor representationsBrain Research Reviews, 2009
- Combined Optimization of Spatial and Temporal Filters for Improving Brain-Computer InterfacingIEEE Transactions on Biomedical Engineering, 2006
- Motor imagery and direct brain-computer communicationProceedings of the IEEE, 2001
- Brain-computer interface technology: a review of the first international meetingIEEE Transactions on Rehabilitation Engineering, 2000
- The neurophysiological basis of motor imageryBehavioural Brain Research, 1996