Selection of Essential Neural Activity Timesteps for Intracortical Brain–Computer Interface Based on Recurrent Neural Network
Open Access
Abstract
Intracortical brain–computer interfaces (iBCIs) translate neural activity into control commands, thereby allowing paralyzed persons to control devices via their brain signals. Recurrent neural networks (RNNs) are widely used as neural decoders because they can learn neural response dynamics from continuous neural activity. Nevertheless, excessively long or short input neural activity for an RNN may decrease its decoding performance. Based on the temporal attention module exploiting relations in features over time, we propose a temporal attention-aware timestep selection (TTS) method that improves the interpretability of the salience of each timestep in an input neural activity. Furthermore, TTS determines the appropriate input neural activity length for accurate neural decoding. Experimental results show that the proposed TTS efficiently selects 28 essential timesteps for RNN-based neural decoders, outperforming state-of-the-art neural decoders on two nonhuman primate datasets ( for monkey Indy and for monkey N). In addition, it reduces the computation time for offline training (reducing 5–12%) and online prediction (reducing 16–18%). When visualizing the attention mechanism in TTS, the preparatory neural activity is consecutively highlighted during arm movement, and the most recent neural activity is highlighted during the resting state in nonhuman primates. Selecting only a few essential timesteps for an RNN-based neural decoder provides sufficient decoding performance and requires only a short computation time.
Keywords
Funding Information
- Ministry of Science and Technology, Taiwan (110-2636-E-006-010 (Young Scholar Fellowship Program), 110-2321-B-010-006, 109-2314-B-303-016, and 109-2221-E-010-004-MY2)
- National Key R&D Program of China (2018YFA0701400)
- National Natural Science Foundation of China (61673346)
- Fundamental Research Funds for the Central Universities (2019XZZX001-01-21)
This publication has 32 references indexed in Scilit:
- LSTM: A Search Space OdysseyIEEE Transactions on Neural Networks and Learning Systems, 2016
- The Largest Response Component in the Motor Cortex Reflects Movement Timing but Not Movement Typeeneuro, 2016
- Principal component analysis: a review and recent developmentsPhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2016
- Clinical translation of a high-performance neural prosthesisNature Medicine, 2015
- A neural network that finds a naturalistic solution for the production of muscle activityNature Neuroscience, 2015
- Optimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex MovementsNeuron, 2014
- Impedance Characterization of Microarray Recording Electrodes in VitroIEEE Transactions on Biomedical Engineering, 2005
- Enhanced Cortical Dopamine Output and Antipsychotic-like Effects of Raclopride by α 2 Adrenoceptor BlockadeScience, 1999
- Long Short-Term MemoryNeural Computation, 1997
- Finding Structure in TimeCognitive Science, 1990