JLJessie R. Liu

Jessie R. Liu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jessie R. Liu has a total of 41 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Simultaneous speech and gesture decoding for multimodal communication in paralysis

    by Samantha C. Brosler, Jessie R. Liu, Alexander B. Silva, Irina P. Hallinan, Cady M. Kurtz-Miott, Jonah F. B. Dunkel Wilker, Adelyn Tu-Chan, Karunesh Ganguly, Edward F. Chang

    Stroke and neurodegenerative diseases can impair speech and nonverbal gestures, limiting natural communication. Brain–computer interfaces (BCIs) aim to restore these functions by translating neural activity into commands for external devices, although prior work has primarily focused on decoding speech or gestures in isolation. Here we show that neural signals recorded with a single high-density electrocorticography implant can support simultaneous decoding of speech and gestures in people with paralysis. We first show that isolated upper-limb and orofacial movements can be reliably decoded among three participants. Using parallel speech and gesture decoders, we then enabled participants to control a personalized virtual avatar by attempting speech and gestures simultaneously or in isolation. Training models on both isolated and simultaneous data improved performance across behavioral contexts. These findings demonstrate that one cortical implant can support multi-effector control and provide a step toward BCIs that enable more natural communication for people with paralysis.


  • Open Access

    Implications of shared motor and perceptual activations on the sensorimotor cortex for neuroprosthetic decoding

    by Alexander Budugur Silva, Jessie R Liu, Vanessa R Anderson, Cady M Kurtz-Miott, Irina P Hallinan, Kaylo T Littlejohn, Samantha Brosler, Adelyn Tu-Chan, Karunesh Ganguly, David Aaron Moses, et al.

    Objective. Neuroprostheses can restore communicative ability to people with paralysis by decoding intended speech motor movements from the sensorimotor cortex (SMC). However, overlapping neural populations in the SMC are also engaged in visual and auditory perceptual processing. The nature of these shared motor and perceptual activations and their potential to interfere with decoding are particularly relevant questions for speech neuroprostheses, as reading and listening are essential daily functions. Approach. In two participants with vocal-tract paralysis and anarthria (ClinicalTrials.gov; NCT03698149), we developed an online electrocorticography (ECoG) based speech-decoding system that maintained accuracy and specificity to intended speech, even during common daily tasks like reading and listening. Offline, we studied the spectrotemporal characteristics and spatial distribution of reading, listening, and attempted-speech responses across our participants’ ECoG arrays. Main results. Across participants, the speech-decoding system had zero false-positive activations during 63.2 minutes of attempted speech and perceptual tasks, maintaining accuracy and specificity to volitional speech attempts. Offline, though we observed shared neural populations that responded to attempted speech, listening, and reading, we found they leveraged different neural representations with differentiable spectrotemporal responses. Shared populations localized to the middle precentral gyrus and may have a distinct role in speech-motor planning. Significance. Potential neuroprosthesis users strongly desire reliable systems that will retain specificity to volitional speech attempts during daily use. These results demonstrate a decoding framework for speech neuroprostheses that maintains this specificity and further our understanding of shared perceptual and motor activity on the SMC.


  • Top Cited Papers

    Speech sequencing in the human precentral gyrus

    by Jessie R. Liu, Lingyun Zhao, Patrick W. Hullett, Edward F. Chang

    Fluent speech production is mediated by serially ordering and preparing motor plans corresponding to target speech sounds, a process known as speech-motor sequencing. Here we used high-density direct cortical recordings while 14 participants spoke utterances with varying phonemic and syllabic sequence complexity after reading a target sequence and a delay period. Phasic activations corresponding to speech production and auditory feedback were observed, but also sustained neural activity that persisted throughout all task phases including the target presentation, the delay period and production of the sequence. Furthermore, sustained activity in a specific area, the middle precentral gyrus (mPrCG), was both modulated by sequence complexity and predicted reaction time, suggesting a role in speech-motor sequencing. Electrocortical stimulation of the mPrCG caused speech disfluencies resembling those seen in apraxia of speech. These results suggest that speech-motor sequencing is mediated by a distributed cortical network in which the mPrCG plays a central role.


  • Open Access

    Implications of shared motor and perceptual activations on the sensorimotor cortex for neuroprosthetic decoding

    by Alexander B. Silva, Jessie R. Liu, Vanessa R. Anderson, Cady M. Kurtz-Miott, Irina P. Hallinan, Kaylo T. Littlejohn, Samantha Brosler, Adelyn Tu-Chan, Karunesh Ganguly, David A. Moses, et al.

    Neuroprostheses can restore communicative ability to people with paralysis by decoding intended speech movements from the sensorimotor cortex (SMC). However, overlapping neural populations in the SMC are also engaged in visual and auditory processing. The nature of these shared motor and perceptual activations and their potential to interfere with decoding are particularly relevant questions for speech neuroprostheses, as reading and listening are essential daily functions. In two participants with vocal-tract paralysis and anarthria (ClinicalTrials.gov; NCT03698149 ), we developed an online electrocorticography (ECoG) based speech-decoding system that maintained accuracy and specificity to intended speech, even during common daily tasks like reading and listening. Offline, we studied the spectrotemporal characteristics and spatial distribution of reading, listening, and attempted-speech responses across our participants’ ECoG arrays. Across participants, the speech-decoding system had zero false-positive activations during 63.2 minutes of attempted speech and perceptual tasks, maintaining accuracy and specificity to volitional speech attempts. Offline, though we observed shared neural populations that responded to attempted speech, listening, and reading, we found they leveraged different neural representations with differentiable spectrotemporal responses. Shared populations localized to the middle precentral gyrus and may have a distinct role in speech-motor planning. Potential neuroprosthesis users strongly desire reliable systems that will retain specificity to volitional speech attempts during daily use. These results demonstrate a decoding framework for speech neuroprostheses that maintains this specificity and further our understanding of shared perceptual and motor activity on the SMC.


  • Top Cited Papers

    A streaming brain-to-voice neuroprosthesis to restore naturalistic communication

    by Kaylo T. Littlejohn, Cheol Jun Cho, Jessie R. Liu, Alexander B. Silva, Bohan Yu, Vanessa R. Anderson, Cady M. Kurtz-Miott, Samantha Brosler, Anshul P. Kashyap, Irina P. Hallinan, et al.

    Natural spoken communication happens instantaneously. Speech delays longer than a few seconds can disrupt the natural flow of conversation. This makes it difficult for individuals with paralysis to participate in meaningful dialogue, potentially leading to feelings of isolation and frustration. Here we used high-density surface recordings of the speech sensorimotor cortex in a clinical trial participant with severe paralysis and anarthria to drive a continuously streaming naturalistic speech synthesizer. We designed and used deep learning recurrent neural network transducer models to achieve online large-vocabulary intelligible fluent speech synthesis personalized to the participant’s preinjury voice with neural decoding in 80-ms increments. Offline, the models demonstrated implicit speech detection capabilities and could continuously decode speech indefinitely, enabling uninterrupted use of the decoder and further increasing speed. Our framework also successfully generalized to other silent-speech interfaces, including single-unit recordings and electromyography. Our findings introduce a speech-neuroprosthetic paradigm to restore naturalistic spoken communication to people with paralysis.


  • Top Cited Papers

    A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages

    by Alexander B. Silva, Jessie R. Liu, Sean L. Metzger, Ilina Bhaya-Grossman, Maximilian E. Dougherty, Margaret P. Seaton, Kaylo T. Littlejohn, Adelyn Tu-Chan, Karunesh Ganguly, David A. Moses, et al.

    Advancements in decoding speech from brain activity have focused on decoding a single language. Hence, the extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear. Here, we leveraged electrocorticography, along with deep-learning and statistical natural-language models of English and Spanish, to record and decode activity from speech-motor cortex of a Spanish–English bilingual with vocal-tract and limb paralysis into sentences in either language. This was achieved without requiring the participant to manually specify the target language. Decoding models relied on shared vocal-tract articulatory representations across languages, which allowed us to build a syllable classifier that generalized across a shared set of English and Spanish syllables. Transfer learning expedited training of the bilingual decoder by enabling neural data recorded in one language to improve decoding in the other language. Overall, our findings suggest shared cortical articulatory representations that persist after paralysis and enable the decoding of multiple languages without the need to train separate language-specific decoders.


  • Top Cited Papers

    The speech neuroprosthesis

    by Alexander B. Silva, Kaylo T. Littlejohn, Jessie R. Liu, David A. Moses, Edward F. Chang

    Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore natural communication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinical epilepsy monitoring and subsequently in individuals with paralysis as part of early feasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapid and substantial progress in decoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.


  • Top Cited Papers
    Open Access

    Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis

    by Sean L. Metzger, Jessie R. Liu, David A. Moses, Maximilian E. Dougherty, Margaret P. Seaton, Kaylo T. Littlejohn, Josh Chartier, Gopala K. Anumanchipalli, Adelyn Tu-Chan, Karunesh Ganguly, et al.

    Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. “alpha” for “a”). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.


  • Long-term retention of ECM hydrogel after implantation into a sub-acute stroke cavity reduces lesion volume

    by Harmanvir Ghuman, Madeline Gerwig, Francesca J. Nicholls, Jessie R. Liu, Julia Donnelly, Stephen F. Badylak, Michel Modo


  • Open Access

    A Comparison of Exogenous Labels for the Histological Identification of Transplanted Neural Stem Cells

    by Francesca J. Nicholls, Jessie R. Liu, Michel Modo

    The interpretation of cell transplantation experiments is often dependent on the presence of an exogenous label for the identification of implanted cells. The exogenous labels Hoechst 33342, 5-bromo-2′-deoxyuridine (BrdU), PKH26, and Qtracker were compared for their labeling efficiency, cellular effects, and reliability to identify a human neural stem cell (hNSC) line implanted intracerebrally into the rat brain. Hoechst 33342 (2 mg/ml) exhibited a delayed cytotoxicity that killed all cells within 7 days. This label was hence not progressed to in vivo studies. PKH26 (5 μM), Qtracker (15 nM), and BrdU (0.2 μM) labeled 100% of the cell population at day 1, although BrdU labeling declined by day 7. BrdU and Qtracker exerted effects on proliferation and differentiation. PKH26 reduced viability and proliferation at day 1, but this normalized by day 7. In an in vitro coculture assay, all labels transferred to unlabeled cells. After transplantation, the reliability of exogenous labels was assessed against the gold standard of a human-specific nuclear antigen (HNA) antibody. BrdU, PKH26, and Qtracker resulted in a very small proportion (<2%) of false positives, but a significant amount of false negatives (~30%), with little change between 1 and 7 days. Exogenous labels can therefore be reliable to identify transplanted cells without exerting major cellular effects, but validation is required. The interpretation of cell transplantation experiments should be presented in the context of the label's limitations.