Multi-Branch Mutual-Distillation Transformer for EEG-Based Seizure Subtype Classification
Open Access
- 13 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 32 (15344320) , 831-839
Abstract
Cross-subject electroencephalogram (EEG) based seizure subtype classification is very important in precise epilepsy diagnostics. Deep learning is a promising solution, due to its ability to automatically extract latent patterns. However, it usually requires a large amount of training data, which may not always be available in clinical practice. This paper proposes Multi-Branch Mutual-Distillation (MBMD) Transformer for cross-subject EEG-based seizure subtype classification, which can be effectively trained from small labeled data. MBMD Transformer replaces all even-numbered encoder blocks of the vanilla Vision Transformer by our designed multi-branch encoder blocks. A mutual-distillation strategy is proposed to transfer knowledge between the raw EEG data and its wavelets of different frequency bands. Experiments on two public EEG datasets demonstrated that our proposed MBMD Transformer outperformed several traditional machine learning and state-of-the-art deep learning approaches. To our knowledge, this is the first work on knowledge distillation for EEG-based seizure subtype classification.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2022YFE0204700)
This publication has 34 references indexed in Scilit:
- EEG datasets for seizure detection and prediction— A reviewEpilepsia Open, 2023
- MixSKD: Self-Knowledge Distillation from Mixup for Image Recognition2022
- A review of feature extraction and performance evaluation in epileptic seizure detection using EEGBiomedical Signal Processing and Control, 2020
- EEGNet: a compact convolutional neural network for EEG-based brain–computer interfacesJournal of Neural Engineering, 2018
- Deep Mutual Learning2018
- On Compressing Deep Models by Low Rank and Sparse Decomposition2017
- Deep Residual Learning for Image Recognition2016
- Feature extraction and recognition of ictal EEG using EMD and SVMComputers in Biology and Medicine, 2013
- Wavelet/mixture of experts network structure for EEG signals classificationExpert Systems with Applications, 2008
- EEG signal classification using wavelet feature extraction and a mixture of expert modelExpert Systems with Applications, 2007