Alexander Banderov

In our previous work, we developed a neuromorphic decoder of intended movements of tetraplegic patients using ECoG recordings from the brain motor cortex, called Motor Control Decoder (MCD). Even though the training data are labeled based on the desired movement, there is no guarantee that the patient is satisfied by the action of the effectors. Hence, the need for the classification of brain signals as satisfactory/unsatisfactory is obvious. Based on previous work, we upgrade our neuromorphic MCD with a Neural Response Decoder (NRD) that is intended to predict whether ECoG data are satisfactory or not in order to improve MCD accuracy. The main aim is to design an actor–critic structure able to adapt via reinforcement learning the MCD (actor) based on NRD (critic) predictions. For this aim, NRD was trained using not only an ECoG signal but also the MCD prediction or prescribed intended movement of the patient. The achieved accuracy of the trained NRD is satisfactory and contributes to improved MCD performance. However, further work has to be carried out to fully utilize the NRD for MCD performance optimization in an on-line manner. Possibility to include feedback from the patient would allow for further improvement of MCD-NRD accuracy.

In our previous work we developed a neuromorphic decoder of intended movements of tetraplegic patients using ECoG recordings from brain motor cortex called Motor Control Decoder (MCD). Even though the training data was labeled based on the desired movement, there is no guarantee that the patient is satisfied by the action of the effectors. Hence the need for classification of brain signals as satisfactory/unsatisfactory is obvious. Based on previous work here we upgrade our neuromorphic MCD with a Neural Response Decoder (NRD) that is intended to predict whether ECoG data is satisfactory or not in order to improve MCD accuracy. The main aim is to design an actor-critic structure able to adapt via reinforcement learning the MCD (actor) based on NRD (critic) predictions. For this aim NRD was trained using not only ECoG signal but also the MCD prediction or prescribed intended movement of the patient. The achieved accuracy of trained NRD is satisfactory and it contributes to improved MCD performance. However, further work has to be done to fully utilize the NRD for MCD performance optimization in on-line manner. Possibility to include feedback from the patient would allow for further improvement of MCD-NRD accuracy.

Current technological solutions for Brain-machine Interfaces (BMI) achieve reasonable accuracy, but most systems are large in size, power consuming and not auto-adaptive. This work addresses the question whether current neuromorphic technologies could resolve these problems? The paper proposes a novel neuromorphic framework of a BMI system for prosthetics control via decoding Electro Cortico-Graphic (ECoG) brain signals. It includes a three-dimensional spike timing neural network (3D-SNN) for brain signals features extraction and an on-line trainable recurrent reservoir structure (Echo state network (ESN)) for Motor Control Decoding (MCD). A software system, written in Python using NEST Simulator SNN library is described. It is able to adapt continuously in real time in supervised or unsupervised mode. The proposed approach was tested on several experimental data sets acquired from a tetraplegic person. First simulation results are encouraging, showing also the need for a further improvement via multiple hyper-parameters tuning. Its future implementation on a neuromorphic hardware platform that is smaller in size and significantly less power consuming is discussed too.

Current technological solutions for Brain-machine Interfaces (BMI) achieve reasonable accuracy, but most systems are large in size, power consuming and not auto-adaptive. This work addresses the question whether current neuromorphic technologies could resolve these problems? The paper proposes a novel neuromorphic framework of a BMI system for prosthetics control via decoding Electro Cortico-Graphic (ECoG) brain signals. It includes a three-dimensional spike timing neural network (3D-SNN) for brain signals features extraction and an on-line trainable recurrent reservoir structure (Echo state network (ESN)) for Motor Control Decoding (MCD). A software system, written in Python using NEST Simulator SNN library is described. It is able to adapt continuously in real time in supervised or unsupervised mode. The proposed approach was tested on several experimental data sets acquired from a tetraplegic person. First simulation results are encouraging, showing also the need for a further improvement via multiple hyper-parameters tuning. Its future implementation on a neuromorphic hardware platform that is smaller in size and significantly less power consuming is discussed too.