Jose M. Carmena
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Jose M. Carmena has a total of 63 co-authors. Here are the top co-authors by number of shared publications:
- Amy L. Orsborn
University of Washington
6 shared publications Karunesh Ganguly
San Francisco VA Medical Center, San Francisco, California, United States of America
6 shared publicationsRyan T. Canolty
Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, California, United States of America
3 shared publicationsCharles F. Cadieu
Helen Wills Neuroscience Institute
2 shared publications- Jan M. Rabaey
EECS Department, University of California, Berkeley, CA, USA
2 shared publications - Jonathan D. Wallis
University of California, Berkeley
2 shared publications
Latest Publications
- 12 June 2022
- Published by Springer Nature
- p. 1474-1487
Functional Neuroscience: Cortical Control of Limb Prostheses
- 12 June 2022
- Published by Springer Nature
- p. 1474-1487
Functional Neuroscience: Cortical Control of Limb Prostheses
- Open Access
- 1 April 2016
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 12 (4) , e1004730
Robust Brain-Machine Interface Design Using Optimal Feedback Control Modeling and Adaptive Point Process Filtering
Much progress has been made in brain-machine interfaces (BMI) using decoders such as Kalman filters and finding their parameters with closed-loop decoder adaptation (CLDA). However, current decoders do not model the spikes directly, and hence may limit the processing time-scale of BMI control and adaptation. Moreover, while specialized CLDA techniques for intention estimation and assisted training exist, a unified and systematic CLDA framework that generalizes across different setups is lacking. Here we develop a novel closed-loop BMI training architecture that allows for processing, control, and adaptation using spike events, enables robust control and extends to various tasks. Moreover, we develop a unified control-theoretic CLDA framework within which intention estimation, assisted training, and adaptation are performed. The architecture incorporates an infinite-horizon optimal feedback-control (OFC) model of the brain’s behavior in closed-loop BMI control, and a point process model of spikes. The OFC model infers the user’s motor intention during CLDA—a process termed intention estimation. OFC is also used to design an autonomous and dynamic assisted training technique. The point process model allows for neural processing, control and decoder adaptation with every spike event and at a faster time-scale than current decoders; it also enables dynamic spike-event-based parameter adaptation unlike current CLDA methods that use batch-based adaptation on much slower adaptation time-scales. We conducted closed-loop experiments in a non-human primate over tens of days to dissociate the effects of these novel CLDA components. The OFC intention estimation improved BMI performance compared with current intention estimation techniques. OFC assisted training allowed the subject to consistently achieve proficient control. Spike-event-based adaptation resulted in faster and more consistent performance convergence compared with batch-based methods, and was robust to parameter initialization. Finally, the architecture extended control to tasks beyond those used for CLDA training. These results have significant implications towards the development of clinically-viable neuroprosthetics. Brain-machine interfaces (BMI) decode subjects’ intended movements from neural activity. Despite significant advances, performance, robustness, and extendibility remain key challenges. Current BMIs use decoders such as Kalman filters whose parameters are estimated in closed-loop BMI operation. However, current decoders do not model the spikes directly and hence may limit the time-scale of neural processing, control, and parameter estimation. Here we develop a novel BMI training architecture for spike-event-based control and parameter estimation, and show that it enables robust control and extends to various tasks. Moreover, we propose a control-theoretic framework for closed-loop decoder training. The BMI incorporates an optimal feedback-control (OFC) model of brain’s control behavior to infer its motor intention. The BMI uses a point process to model the spikes; this enables the subject to control the movement and the decoder to estimate the parameters using spike events. Using closed-loop experiments in a non-human primate, we show that the OFC model improves performance compared with current intention estimation techniques; that spike-event-based adaptation enables faster performance convergence compared with current batch-based methods, and is robust to parameter initialization; and that the architecture extends to various tasks. This architecture has significant implications for future clinically-viable BMIs.
- 1 April 2016
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 12 (4) , e1004730
Open AccessRobust Brain-Machine Interface Design Using Optimal Feedback Control Modeling and Adaptive Point Process Filtering
Much progress has been made in brain-machine interfaces (BMI) using decoders such as Kalman filters and finding their parameters with closed-loop decoder adaptation (CLDA). However, current decoders do not model the spikes directly, and hence may limit the processing time-scale of BMI control and adaptation. Moreover, while specialized CLDA techniques for intention estimation and assisted training exist, a unified and systematic CLDA framework that generalizes across different setups is lacking. Here we develop a novel closed-loop BMI training architecture that allows for processing, control, and adaptation using spike events, enables robust control and extends to various tasks. Moreover, we develop a unified control-theoretic CLDA framework within which intention estimation, assisted training, and adaptation are performed. The architecture incorporates an infinite-horizon optimal feedback-control (OFC) model of the brain’s behavior in closed-loop BMI control, and a point process model of spikes. The OFC model infers the user’s motor intention during CLDA—a process termed intention estimation. OFC is also used to design an autonomous and dynamic assisted training technique. The point process model allows for neural processing, control and decoder adaptation with every spike event and at a faster time-scale than current decoders; it also enables dynamic spike-event-based parameter adaptation unlike current CLDA methods that use batch-based adaptation on much slower adaptation time-scales. We conducted closed-loop experiments in a non-human primate over tens of days to dissociate the effects of these novel CLDA components. The OFC intention estimation improved BMI performance compared with current intention estimation techniques. OFC assisted training allowed the subject to consistently achieve proficient control. Spike-event-based adaptation resulted in faster and more consistent performance convergence compared with batch-based methods, and was robust to parameter initialization. Finally, the architecture extended control to tasks beyond those used for CLDA training. These results have significant implications towards the development of clinically-viable neuroprosthetics. Brain-machine interfaces (BMI) decode subjects’ intended movements from neural activity. Despite significant advances, performance, robustness, and extendibility remain key challenges. Current BMIs use decoders such as Kalman filters whose parameters are estimated in closed-loop BMI operation. However, current decoders do not model the spikes directly and hence may limit the time-scale of neural processing, control, and parameter estimation. Here we develop a novel BMI training architecture for spike-event-based control and parameter estimation, and show that it enables robust control and extends to various tasks. Moreover, we propose a control-theoretic framework for closed-loop decoder training. The BMI incorporates an optimal feedback-control (OFC) model of brain’s control behavior to infer its motor intention. The BMI uses a point process to model the spikes; this enables the subject to control the movement and the decoder to estimate the parameters using spike events. Using closed-loop experiments in a non-human primate, we show that the OFC model improves performance compared with current intention estimation techniques; that spike-event-based adaptation enables faster performance convergence compared with current batch-based methods, and is robust to parameter initialization; and that the architecture extends to various tasks. This architecture has significant implications for future clinically-viable BMIs.
- 24 March 2015
- Published by Springer Nature
- p. 1236-1249
Functional Neuroscience: Cortical Control of Limb Prosthesis
- 24 March 2015
- Published by Springer Nature
- p. 1236-1249
Functional Neuroscience: Cortical Control of Limb Prosthesis
- Open Access
Disentangling Multidimensional Spatio-Temporal Data into their Common and Aberrant Responses
With the advent of high-throughput measurement techniques, scientists and engineers are starting to grapple with massive data sets and encountering challenges with how to organize, process and extract information into meaningful structures. Multidimensional spatio-temporal biological data sets such as time series gene expression with various perturbations with different cell lines, or neural spike data sets across many experimental trials have the potential to acquire insight across multiple dimensions. For this potential to be realized, we need a suitable representation to turn data into insight. Since a wide range of experiments and the (unknown) complexity of underlying system make biological data more heterogeneous than those in other fields, we propose the method based on Robust Principal Component Analysis (RPCA), which is well suited for extracting principal components where we have corrupted observations. The proposed method provides us a new representation of these data sets which consists of its common and aberrant response. This representation might help users to acquire a new insight from data. %For example, identifying common event-related neural features across many experimental trials can be used as a signature to detect discrete events or state transitions. Also, the proposed method can be useful to biologists in clustering and analyzing gene expression time series data with a new perspective, for example, it can not only extract canonical cell signaling response but also inform them to get insight into the heterogeneity of different responses across different cell lines.
Open AccessDisentangling Multidimensional Spatio-Temporal Data into their Common and Aberrant Responses
With the advent of high-throughput measurement techniques, scientists and engineers are starting to grapple with massive data sets and encountering challenges with how to organize, process and extract information into meaningful structures. Multidimensional spatio-temporal biological data sets such as time series gene expression with various perturbations with different cell lines, or neural spike data sets across many experimental trials have the potential to acquire insight across multiple dimensions. For this potential to be realized, we need a suitable representation to turn data into insight. Since a wide range of experiments and the (unknown) complexity of underlying system make biological data more heterogeneous than those in other fields, we propose the method based on Robust Principal Component Analysis (RPCA), which is well suited for extracting principal components where we have corrupted observations. The proposed method provides us a new representation of these data sets which consists of its common and aberrant response. This representation might help users to acquire a new insight from data. %For example, identifying common event-related neural features across many experimental trials can be used as a signature to detect discrete events or state transitions. Also, the proposed method can be useful to biologists in clustering and analyzing gene expression time series data with a new perspective, for example, it can not only extract canonical cell signaling response but also inform them to get insight into the heterogeneity of different responses across different cell lines.
- Open Access
- 5 March 2014
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 8, 20
Muscle synergies evoked by microstimulation are preferentially encoded during behavior
Electrical microstimulation studies provide some of the most direct evidence for the neural representation of muscle synergies. These synergies, i.e. coordinated activations of groups of muscles, have been proposed as building blocks for the construction of motor behaviors by the nervous system. Intraspinal or intracortical microstimulation has been shown to evoke muscle patterns that can be resolved into a small set of synergies similar to those seen in natural behavior. However, questions remain about the validity of microstimulation as a probe of neural function, particularly given the relatively long trains of supratheshold stimuli used in these studies. Here, we examined whether muscle synergies evoked during intracortical microstimulation in two rhesus macaques were similarly encoded by nearby motor cortical units during a purely voluntary behavior involving object reach, grasp, and carry movements. At each microstimulation site we identified the synergy most strongly evoked among those extracted from muscle patterns evoked over all microstimulation sites. For each cortical unit recorded at the same microstimulation site, we then identified the synergy most strongly encoded among those extracted from muscle patterns recorded during the voluntary behavior. We found that the synergy most strongly evoked at an intracortical microstimulation site matched the synergy most strongly encoded by proximal units more often than expected by chance. These results suggest a common neural substrate for microstimulation-evoked motor responses and for the generation of muscle patterns during natural behaviors.
- 5 March 2014
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 8, 20
Open AccessMuscle synergies evoked by microstimulation are preferentially encoded during behavior
Electrical microstimulation studies provide some of the most direct evidence for the neural representation of muscle synergies. These synergies, i.e. coordinated activations of groups of muscles, have been proposed as building blocks for the construction of motor behaviors by the nervous system. Intraspinal or intracortical microstimulation has been shown to evoke muscle patterns that can be resolved into a small set of synergies similar to those seen in natural behavior. However, questions remain about the validity of microstimulation as a probe of neural function, particularly given the relatively long trains of supratheshold stimuli used in these studies. Here, we examined whether muscle synergies evoked during intracortical microstimulation in two rhesus macaques were similarly encoded by nearby motor cortical units during a purely voluntary behavior involving object reach, grasp, and carry movements. At each microstimulation site we identified the synergy most strongly evoked among those extracted from muscle patterns evoked over all microstimulation sites. For each cortical unit recorded at the same microstimulation site, we then identified the synergy most strongly encoded among those extracted from muscle patterns recorded during the voluntary behavior. We found that the synergy most strongly evoked at an intracortical microstimulation site matched the synergy most strongly encoded by proximal units more often than expected by chance. These results suggest a common neural substrate for microstimulation-evoked motor responses and for the generation of muscle patterns during natural behaviors.
- 26 September 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Antennas and Wireless Propagation Letters
- Vol. 12, 1307-1310
Measurement of Wireless Link for Brain–Machine Interface Systems Using Human-Head Equivalent Liquid
In this letter, we analyze the wireless power transfer from an on-body transmit antenna to a millimeter-size antenna in a cortical implant. The studied wireless link provides power and data telemetry for a battery-free wireless brain-machine interface microelectronic system. We present a cubic 2 × 2 × 2 mm3 implant loop and analyze the effect of a magneto-dielectric core material on the performance of the antenna. We compare the wireless link performances to two different transmit antennas: solid and 2-segmented loops. Our simulation results show that the 2-segmented loop provides reduced near electric field, but maintains the link power efficiency. Finally, we validate the simulated link power efficiencies through measurements in air and in human-head equivalent liquid at 300 MHz.
- 26 September 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Antennas and Wireless Propagation Letters
- Vol. 12, 1307-1310
Measurement of Wireless Link for Brain–Machine Interface Systems Using Human-Head Equivalent Liquid
In this letter, we analyze the wireless power transfer from an on-body transmit antenna to a millimeter-size antenna in a cortical implant. The studied wireless link provides power and data telemetry for a battery-free wireless brain-machine interface microelectronic system. We present a cubic 2 × 2 × 2 mm3 implant loop and analyze the effect of a magneto-dielectric core material on the performance of the antenna. We compare the wireless link performances to two different transmit antennas: solid and 2-segmented loops. Our simulation results show that the 2-segmented loop provides reduced near electric field, but maintains the link power efficiency. Finally, we validate the simulated link power efficiencies through measurements in air and in human-head equivalent liquid at 300 MHz.
- Open Access
- 21 May 2013
- Published by Public Library of Science (PLoS) in PLOS Biology
- Vol. 11 (5) , e1001561
Advances in Neuroprosthetic Learning and Control
Significant progress has occurred in the field of brain–machine interfaces (BMI) since the first demonstrations with rodents, monkeys, and humans controlling different prosthetic devices directly with neural activity. This technology holds great potential to aid large numbers of people with neurological disorders. However, despite this initial enthusiasm and the plethora of available robotic technologies, existing neural interfaces cannot as yet master the control of prosthetic, paralyzed, or otherwise disabled limbs. Here I briefly discuss recent advances from our laboratory into the neural basis of BMIs that should lead to better prosthetic control and clinically viable solutions, as well as new insights into the neurobiology of action.
- 21 May 2013
- Published by Public Library of Science (PLoS) in PLOS Biology
- Vol. 11 (5) , e1001561
Open AccessAdvances in Neuroprosthetic Learning and Control
Significant progress has occurred in the field of brain–machine interfaces (BMI) since the first demonstrations with rodents, monkeys, and humans controlling different prosthetic devices directly with neural activity. This technology holds great potential to aid large numbers of people with neurological disorders. However, despite this initial enthusiasm and the plethora of available robotic technologies, existing neural interfaces cannot as yet master the control of prosthetic, paralyzed, or otherwise disabled limbs. Here I briefly discuss recent advances from our laboratory into the neural basis of BMIs that should lead to better prosthetic control and clinically viable solutions, as well as new insights into the neurobiology of action.
- Open Access
- 15 May 2013
- Published by American Physiological Society in Journal of Neurophysiology
- Vol. 109 (10) , 2585-2595
Dynamic changes of rodent somatosensory barrel cortex are correlated with learning a novel conditioned stimulus
The rodent somatosensory barrel cortex (S1bf) has proved a valuable model for studying neural plasticity in vivo. It has been observed that sensory deprivation or conditioning reorganizes sensory-driven activity within S1bf. These observations suggest a role for S1bf in somatosensory learning. This study evaluated the hypothesis that the response properties of extracellularly recorded neurons in S1bf would change as subjects learned to respond to stimulation of S1bf. Intracortical microstimulation (ICMS) of S1bf was used as a means for bypassing feedforward drive from the sensory periphery, midbrain, and thalamus while exciting local cortical networks. To separate the learning of this conditioned stimulus-conditioned response (CS-CR) from other elements of the task, we employed a cross-modal transfer schedule. Long-Evans rats were initially trained to respond to an auditory stimulus. All subjects were then implanted in both S1bfs with chronic microwire arrays for recording neural activity and delivering ICMS. Next, this association was transferred to ICMS of one hemisphere's S1bf. S1bf responded to ICMS with a brief increase in firing rate followed by a longer reduction in activity. We observed that the duration of reduced activity elicited by ICMS increased as the subjects began to respond correctly more often than expected by chance, and the magnitude of the initial positive response increased as they consolidated this CS-CR. Subsequent ICMS of the opposite S1bf revealed that this CS-CR did not generalize across hemispheres. These results suggest that a mechanism involving a single hemisphere's S1bf tunes cortical responses in concert with changes in rodent behavior during somatosensory learning.
- 15 May 2013
- Published by American Physiological Society in Journal of Neurophysiology
- Vol. 109 (10) , 2585-2595
Open AccessDynamic changes of rodent somatosensory barrel cortex are correlated with learning a novel conditioned stimulus
The rodent somatosensory barrel cortex (S1bf) has proved a valuable model for studying neural plasticity in vivo. It has been observed that sensory deprivation or conditioning reorganizes sensory-driven activity within S1bf. These observations suggest a role for S1bf in somatosensory learning. This study evaluated the hypothesis that the response properties of extracellularly recorded neurons in S1bf would change as subjects learned to respond to stimulation of S1bf. Intracortical microstimulation (ICMS) of S1bf was used as a means for bypassing feedforward drive from the sensory periphery, midbrain, and thalamus while exciting local cortical networks. To separate the learning of this conditioned stimulus-conditioned response (CS-CR) from other elements of the task, we employed a cross-modal transfer schedule. Long-Evans rats were initially trained to respond to an auditory stimulus. All subjects were then implanted in both S1bfs with chronic microwire arrays for recording neural activity and delivering ICMS. Next, this association was transferred to ICMS of one hemisphere's S1bf. S1bf responded to ICMS with a brief increase in firing rate followed by a longer reduction in activity. We observed that the duration of reduced activity elicited by ICMS increased as the subjects began to respond correctly more often than expected by chance, and the magnitude of the initial positive response increased as they consolidated this CS-CR. Subsequent ICMS of the opposite S1bf revealed that this CS-CR did not generalize across hemispheres. These results suggest that a mechanism involving a single hemisphere's S1bf tunes cortical responses in concert with changes in rodent behavior during somatosensory learning.
- Open Access
Function Identification in Neuron Populations via Information Bottleneck
It is plausible to hypothesize that the spiking responses of certain neurons represent functions of the spiking signals of other neurons. A natural ensuing question concerns how to use experimental data to infer what kind of a function is being computed. Model-based approaches typically require assumptions on how information is represented. By contrast, information measures are sensitive only to relative behavior: information is unchanged by applying arbitrary invertible transformations to the involved random variables. This paper develops an approach based on the information bottleneck method that attempts to find such functional relationships in a neuron population. Specifically, the information bottleneck method is used to provide appropriate compact representations which can then be parsed to infer functional relationships. In the present paper, the parsing step is specialized to the case of remapped-linear functions. The approach is validated on artificial data and then applied to recordings from the motor cortex of a macaque monkey performing an arm-reaching task. Functional relationships are identified and shown to exhibit some degree of persistence across multiple trials of the same experiment.
Open AccessFunction Identification in Neuron Populations via Information Bottleneck
It is plausible to hypothesize that the spiking responses of certain neurons represent functions of the spiking signals of other neurons. A natural ensuing question concerns how to use experimental data to infer what kind of a function is being computed. Model-based approaches typically require assumptions on how information is represented. By contrast, information measures are sensitive only to relative behavior: information is unchanged by applying arbitrary invertible transformations to the involved random variables. This paper develops an approach based on the information bottleneck method that attempts to find such functional relationships in a neuron population. Specifically, the information bottleneck method is used to provide appropriate compact representations which can then be parsed to infer functional relationships. In the present paper, the parsing step is specialized to the case of remapped-linear functions. The approach is validated on artificial data and then applied to recordings from the motor cortex of a macaque monkey performing an arm-reaching task. Functional relationships are identified and shown to exhibit some degree of persistence across multiple trials of the same experiment.
- Open Access
- 1 January 2013
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 7, 157
Creating new functional circuits for action via brain-machine interfaces
Brain-machine interfaces (BMIs) are an emerging technology with great promise for developing restorative therapies for those with disabilities. BMIs also create novel, well-defined functional circuits for action that are distinct from the natural sensorimotor apparatus. Closed-loop control of BMI systems can also actively engage learning and adaptation. These properties make BMIs uniquely suited to study learning of motor and non-physical, abstract skills. Recent work used motor BMIs to shed light on the neural representations of skill formation and motor adaptation. Emerging work in sensory BMIs, and other novel interface systems, also highlight the promise of using BMI systems to study fundamental questions in learning and sensorimotor control. This paper outlines the interpretation of BMIs as novel closed-loop systems and the benefits of these systems for studying learning. We review BMI learning studies, their relation to motor control, and propose future directions for this nascent field. Understanding learning in BMIs may both elucidate mechanisms of natural motor and abstract skill learning, and aid in developing the next generation of neuroprostheses.
- 1 January 2013
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 7, 157
Open AccessCreating new functional circuits for action via brain-machine interfaces
Brain-machine interfaces (BMIs) are an emerging technology with great promise for developing restorative therapies for those with disabilities. BMIs also create novel, well-defined functional circuits for action that are distinct from the natural sensorimotor apparatus. Closed-loop control of BMI systems can also actively engage learning and adaptation. These properties make BMIs uniquely suited to study learning of motor and non-physical, abstract skills. Recent work used motor BMIs to shed light on the neural representations of skill formation and motor adaptation. Emerging work in sensory BMIs, and other novel interface systems, also highlight the promise of using BMI systems to study fundamental questions in learning and sensorimotor control. This paper outlines the interpretation of BMIs as novel closed-loop systems and the benefits of these systems for studying learning. We review BMI learning studies, their relation to motor control, and propose future directions for this nascent field. Understanding learning in BMIs may both elucidate mechanisms of natural motor and abstract skill learning, and aid in developing the next generation of neuroprostheses.