Study of a Multi-modal Neurorobotic Prosthetic Arm Control System based on Recurrent Spiking Neural Network
Open Access
- 13 May 2022
- journal article
- research article
- Published by EDP Sciences in SHS Web of Conferences
- Vol. 139, 03019
Abstract
The use of robotic arms in various fields of human endeavor has increased over the years, and with recent advancements in artificial intelligence enabled by deep learning, they are increasingly being employed in medical applications like assistive robots for paralyzed patients with neurological disorders, welfare robots for the elderly, and prosthesis for amputees. However, robot arms tailored towards such applications are resource-constrained. As a result, deep learning with conventional artificial neural network (ANN) which is often run on GPU with high computational complexity and high power consumption cannot be handled by them. Neuromorphic processors, on the other hand, leverage spiking neural network (SNN) which has been shown to be less computationally complex and consume less power, making them suitable for such applications. Also, most robot arms unlike living agents that combine different sensory data to accurately perform a complex task, use uni-modal data which affects their accuracy. Conversely, multi-modal sensory data has been demonstrated to reach high accuracy and can be employed to achieve high accuracy in such robot arms. This paper presents the study of a multi-modal neurorobotic prosthetic arm control system based on recurrent spiking neural network. The robot arm control system uses multi-modal sensory data from visual (camera) and electromyography sensors, together with spike-based data processing on our previously proposed R-NASH neuromorphic processor to achieve robust accurate control of a robot arm with low power. The evaluation result using both uni-modal and multi-modal input data show that the multi-modal input achieves a more robust performance at 87%, compared to the uni-modal.Keywords
This publication has 24 references indexed in Scilit:
- Deep Multimodal Representation Learning: A SurveyIEEE Access, 2019
- Probabilistic Encoding Models for Multivariate Neural DataFrontiers in Neural Circuits, 2019
- Robotic Arm Based Automatic Ultrasound Scanning for Three-Dimensional ImagingIEEE Transactions on Industrial Informatics, 2018
- Image Rectification Software Test Automation Using a Robotic ARMIEEE Access, 2018
- Control of Omnidirectional Robot Using Z-Number-Based Fuzzy SystemIEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018
- Gesture Controlled Mobile Robotic Arm for Elderly and Wheelchair People Assistance Using Kinect Sensor2018
- A neuromorphic controller for a robotic vehicle equipped with a dynamic vision sensor2017
- Non-Invasive Stimulation-Based Tactile Sensation for Upper-Extremity Prosthesis: A ReviewIEEE Sensors Journal, 2017
- Potential Advantages Using Large Anthropomorphic Robots in Human-robot Collaborative, Hand Guided AssemblyProcedia CIRP, 2016
- Hearing lips and seeing voicesNature, 1976