The challenge of out-of-distribution detection in motor imagery BCIs
Open Access
- 1 October 2026
- journal article
- Published by IOP Publishing in Journal of Neural Engineering
- Vol. 23 (5) , 056046
Abstract
Objective . Machine learning classifiers used in Brain-Computer Interfaces (BCIs) make classifications based on the distribution of examples on which they were trained. When exposed to electroencephalography (EEG) from unfamiliar classes they can only make blind guesses. Instead of allowing such guesses, these out-of-distribution (OOD) samples should be detected and rejected to improve the robustness of BCIs against unfamiliar cognitive states. Approach . We study OOD detection in Motor Imagery BCIs by training a model on some classes and observing whether an unfamiliar movement class can be detected based on increased uncertainty. We tested seven different OOD detection methods and one more method that has been claimed to boost the quality of OOD detection. Main results . For many BCI users, the uncertainty for the familiar in-distribution classes can still be higher than for the OOD classes, due to the high intrinsic variability inherent in EEG signals. As a result, many OOD detection methods that have shown good performance in other machine learning domains prove to be ineffective at identifying unfamiliar motor imagery patterns in BCIs. However, we also found that OOD detection performance is correlated with on-task performance, and that Deep Ensemble models and Monte Carlo-Dropout models were able to achieve on-task Area Under the Receiver-Operator Characteristic curve > 0.9, and OOD detection ability up to 0.7. This shows that for models and subjects where task performance is high, rejecting unfamiliar cognitive states becomes feasible. Significance . Our research demonstrates a Leave-One-Class-Out OOD detection setup as an experimental paradigm for studying whether models are robust against unfamiliar motor imagery actions that were not seen in the training data. This is the first benchmark in Motor Imagery BCI that evaluates this type of OOD data. Our results demonstrate how to improve the overall safety and reliability of BCIs by preventing erroneous actions.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Uncertainty Quantification in Machine Learning for Biosignal Applications - A ReviewJournal of Healthcare Informatics Research, 2026
- Uncertainty in deep learning for EEG under dataset shiftsArtificial Intelligence in Medicine, 2026
- Long-term unsupervised recalibration of cursor-based intracortical brain–computer interfaces using a hidden Markov modelNature Biomedical Engineering, 2025
- Generalized Out-of-Distribution Detection: A SurveyInternational Journal of Computer Vision, 2024
- Toward reliable signals decoding for electroencephalogram: A benchmark study to EEGNeXBiomedical Signal Processing and Control, 2023
- UNCER: A framework for uncertainty estimation and reduction in neural decoding of EEG signalsNeurocomputing, 2023
- Deep learning with convolutional neural networks for EEG decoding and visualizationHuman Brain Mapping, 2017
- Motor imagery based brain–computer interfaces: An emerging technology to rehabilitate motor deficitsNeuropsychologia, 2015
- Motor imagery-based brain activity parallels that of motor execution: Evidence from magnetic source imaging of cortical oscillationsBrain Research, 2014
- Equivalent is not equal: Primary motor cortex (MI) activation during motor imagery and execution of sequential movementsBrain Research, 2008