BRAND: A platform for closed-loop experiments with deep network models
Authors:
- Yahia H. AliYAYahia H. Ali
- Kevin BodkinKBKevin Bodkin
- Mattia Rigotti-ThompsonMRMattia Rigotti-Thompson
- Nicholas S. CardNCNicholas S. Card
- Bareesh BhaduriBBBareesh Bhaduri
- Samuel R. Nason-TomaszewskiSNSamuel R. Nason-Tomaszewski
- Domenick M. MifsudDMDomenick M. Mifsud
- Xianda HouXHXianda Hou
- Claire NicolasCNClaire Nicolas
- Shane AllcroftSAShane Allcroft
- Leigh R. HochbergLHLeigh R. Hochberg
Preprint
Abstract
Artificial neural networks (ANNs) are state-of-the-art tools for modeling and decoding neural activity, but deploying them in closed-loop experiments with tight timing constraints is challenging due to their limited support in existing real-time frameworks. Researchers need a platform that fully supports high-level languages for running ANNs (e.g., Python and Julia) while maintaining support for languages that are critical for low-latency data acquisition and processing (e.g., C and C++). To address these needs, we introduce the Backend for Realtime Asynchronous Neural Decoding (BRAND). BRAND comprises Linux processes, termed nodes , which communicate with each other in a graph via streams of data. Its asynchronous design allows for acquisition, control, and analysis to be executed in parallel on streams of data that may operate at different timescales. BRAND uses Redis to send data between nodes, which enables fast inter-process communication and supports 54 different programming languages. Thus, developers can easily deploy existing ANN models in BRAND with minimal implementation changes. In our tests, BRAND achieved <600 microsecond latency between processes when sending large quantities of data (1024 channels of 30 kHz neural data in 1-millisecond chunks). BRAND runs a brain-computer interface with a recurrent neural network (RNN) decoder with less than 8 milliseconds of latency from neural data input to decoder prediction. In a real-world demonstration of the system, participant T11 in the BrainGate2 clinical trial performed a standard cursor control task, in which 30 kHz signal processing, RNN decoding, task control, and graphics were all executed in BRAND. This system also supports real-time inference with complex latent variable models like Latent Factor Analysis via Dynamical Systems. By providing a framework that is fast, modular, and language-agnostic, BRAND lowers the barriers to integrating the latest tools in neuroscience and machine learning into closed-loop experiments.Keywords
All Related Versions
- Published version: Journal of Neural Engineering, 21 (2), 026046.
This publication has 36 references indexed in Scilit:
- Bonsai: an event-based framework for processing and controlling data streamsFrontiers in Neuroinformatics, 2015
- High-performance neuroprosthetic control by an individual with tetraplegiaThe Lancet, 2013
- A high-performance neural prosthesis enabled by control algorithm designNature Neuroscience, 2012
- Topological domains in mammalian genomes identified by analysis of chromatin interactionsNature, 2012
- Quickest detection of drug-resistant seizures: An optimal control approachEpilepsy & Behavior, 2011
- A closed-loop human simulator for investigating the role of feedback control in brain-machine interfacesJournal of Neurophysiology, 2011
- Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode arrayJournal of Neural Engineering, 2011
- BCI2000: A General-Purpose Brain-Computer Interface (BCI) SystemIEEE Transactions on Biomedical Engineering, 2004
- Direct Cortical Control of 3D Neuroprosthetic DevicesScience, 2002
- Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentencesIEEE Transactions on Acoustics, Speech, and Signal Processing, 1980