SHStephanie Haro

Stephanie Haro

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Stephanie Haro has a total of 38 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Sentence-level neural dynamics in speech motor cortex

    by H. Levi-Aharoni, J. J. Jude, S. Haro, A. J. Acosta, S. B. Allcroft, M. C. Ottenhoff, N. S. Card, M. Wairagkar, C. Iacobacci, A. L. Pritchard, et al.

    Prior recordings from speech motor cortex have revealed phonemic, syllabic, articulatory and speech-sequence representations, while more recent studies indicate that activity may also reflect higher-order features of speech production. However, whether speech motor cortex is sensitive to the linguistic organization of sentences remains unclear. Here we show that single-unit activity in speech motor cortex contains information about sentence-level linguistic structure. In three BrainGate2 participants with implanted microelectrode arrays in the left precentral gyrus, we recorded neural activity during an instructed-delay speech task comprising English sentences, word lists, syntactically structured nonword sentences and nonword lists. Neural activity in areas 6v and 55b contained speech- and language-related components, with stronger stimulus-condition effects in 55b, and stimulus condition was decodable in all three participants. In two larger datasets, semantic categories of English sentences were decodable from neural activity. These findings demonstrate that speech motor cortex encodes information about linguistic structure and semantic content at the level of sentences.

  • Open Access

    A Brain-Computer Interface for Improving Auditory Attention in Multi-Talker Environments

    by Stephanie Haro, Christine Beauchene, Thomas F. Quatieri, Christopher J. Smalt

    There is significant research in accurately determining the focus of a listener’s attention in a multi-talker environment using auditory attention decoding (AAD) algorithms. These algorithms rely on neural signals to identify the intended speaker, assuming that these signals consistently reflect the listener’s focus. However, some listeners struggle with this competing talkers task, leading to suboptimal tracking of the desired speaker due to potential interference from distractors. The goal of this study was to enhance a listener’s attention to the target speaker in real time and investigate the underlying neuralbn bases of this improvement. This paper describes a closed-loop neurofeedback system that decodes the auditory attention of the listener in real time, utilizing data from a non-invasive, wet electroencephalography (EEG) brain-computer interface (BCI). Fluctuations in the listener’s real-time attention decoding accuracy was used to provide acoustic feedback. As accuracy improved, the ignored talker in the two-talker listening scenario was attenuated; making the desired talker easier to attend to due to the improved attended talker signal-to-noise ratio (SNR). A one-hour session was divided into a 10-minute decoder training phase, with the rest of the session allocated to observing changes in neural decoding. In this study, we found evidence of suppression of (i.e., reduction in) net neural tracking and decoding of the unattended talker when comparing the first and second half of the neurofeedback session ((p = 0.02, Cohen’s d = −1.29, 95% CI [−0.02,−0.01] and p = 0.01, Cohen’s d = −1.56, 95% CI [−7.25,−3.44], respectively). We did not find a statistically significant increase in the neural tracking or decoding of the attended talker. These results establish a single session performance benchmark for a time-invariant, non-adaptive attended talker linear decoder utilized to extract attention from a listener integrated within a closed-loop neurofeedback system. This research lays the engineering and scientific foundation for prospective multi-session clinical trials of an auditory attention training paradigm.


  • Open Access

    Decoding intended speech with an intracortical brain-computer interface in a person with longstanding anarthria and locked-in syndrome

    by Justin J. Jude, Stephanie Haro, Hadar Levi-Aharoni, Hiroaki Hashimoto, Alexander J. Acosta, Nicholas S. Card, Maitreyee Wairagkar, David M. Brandman, Sergey D. Stavisky, Ziv M. Williams, et al.

    Intracortical brain-computer interfaces (iBCIs) for decoding intended speech have provided individuals with ALS and severe dysarthria an intuitive method for high-throughput communication. These advances have been demonstrated in individuals who are still able to vocalize and move speech articulators. Here, we decoded intended speech from an individual with longstanding anarthria, locked-in syndrome, and ventilator dependence due to advanced symptoms of ALS. We found that phonemes, words, and higher-order language units could be decoded well above chance. While sentence decoding accuracy was below that of demonstrations in participants with dysarthria, we are able to attain an extensive characterization of the neural signals underlying speech in a person with locked-in syndrome and through our results identify several directions for future improvement. These include closed-loop speech imagery training and decoding linguistic (rather than phonemic) units from neural signals in middle precentral gyrus. Overall, these results demonstrate that speech decoding from motor cortex may be feasible in people with anarthria and ventilator dependence. For individuals with longstanding anarthria, a purely phoneme-based decoding approach may lack the accuracy necessary to support independent use as a primary means of communication; however, additional linguistic information embedded within neural signals may provide a route to augment the performance of speech decoders.


  • Open Access

    A Brain-Computer Interface for Improving Auditory Attention in Multi-Talker Environments

    by S Haro, C Beauchene, T F Quatieri, C J Smalt

    Objective: There is significant research in accurately determining the focus of a listener’s attention in a multi-talker environment using auditory attention decoding (AAD) algorithms. These algorithms rely on neural signals to identify the intended speaker, assuming that these signals consistently reflect the listener’s focus. However, some listeners struggle with this competing talkers task, leading to suboptimal tracking of the desired speaker due to potential interference from distractors. The goal of this study was to enhance a listener’s attention to the target speaker in real time and investigate the underlying neural bases of this improvement. Approach: This paper describes a closed-loop neurofeedback system that decodes the auditory attention of the listener in real time, utilizing data from a non-invasive, wet electroencephalography (EEG) brain-computer interface (BCI). Fluctuations in the listener’s real-time attention decoding accuracy was used to provide acoustic feedback. As accuracy improved, the ignored talker in the two-talker listening scenario was attenuated; making the desired talker easier to attend to due to the improved attended talker signal-to-noise ratio (SNR). A one-hour session was divided into a 10-minute decoder training phase, with the rest of the session allocated to observing changes in neural decoding. Results: In this study, we found evidence of suppression of (i.e., reduction in) neural tracking of the unattended talker when comparing the first and second half of the neurofeedback session ( p = 0.012). We did not find a statistically significant increase in the neural tracking of the attended talker. Significance: These results establish a single session performance benchmark for a time-invariant, non-adaptive attended talker linear decoder utilized to extract attention from a listener integrated within a closed-loop neurofeedback system. This research lays the engineering and scientific foundation for prospective multi-session clinical trials of an auditory attention training paradigm.


  • Open Access

    EEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort

    by Stephanie Haro, Hrishikesh M. Rao, Thomas F. Quatieria, Christopher J. Smalt

    Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial‐audio cues. A least‐squares, electroencephalography (EEG)‐based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5‐s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5‐s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [ ] and minimum parietal alpha power measure [ ]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization ( ). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech‐based features.


  • Open Access

    EEG Alpha Power and Pupil Diameter Reflect Endogenous Auditory Attention Switching and Listening Effort

    by Stephanie Haro, Hrishikesh M. Rao, Thomas F. Quatieri, Christopher J. Smalt

    Auditory attention describes a listeners focus on an acoustic source while they ignore other competing sources that might be present. In an environment with multiple talkers and background noise (i.e. the cocktail party effect), auditory attention can be difficult, requiring the listener to expend measurable cognitive effort. A listener will naturally interrupt sustained attention on a source when switching towards another source during conversation. This change in attention is potentially even more taxing than maintaining sustained attention due to the limits of human working memory, and this additional effort required has not been well studied. In this work, we evaluated an attention decoder algorithm for detecting the change in attention and investigated cognitive effort expended during attentional switching and sustained attention. Two variants of endogenous attention switching were explored: the switches either had in-the-moment decision making or a pre-defined attentional switch time. A least-squares, EEG-based, attention decoding algorithm achieved 64.1% accuracy with a 5-second correlation window and illustrated smooth transitions in the attended talker prediction through switches in sustained attention at approximately half of the analysis window size (2.2 seconds). The expended listening effort, as measured by simultaneous electroencephalography (EEG) and pupillometry, was also a strong indicator of switching. Specifically, centrotemporal alpha power [F(2, 18) = 7.473, P = 0.00434] and mean pupil diameter [F(2, 18) = 9.159, P = 0.0018] were significantly different for trials that contained a switch in comparison to sustained trials. We also found that relative attended and ignored talker locations modulate the EEG alpha topographic response. This alpha lateralization was found to be impacted by the interaction between experimental condition and whether the measure was computed before or after the switch [F(2,18) = 3.227, P = 0.0634]. These results suggest that expended listening effort is a promising feature that should be pursued in a decoding context, in addition to speech and location-based features.


  • Open Access

    A comprehensive computational model of animal biosonar signal processing

    by Chen Ming, Stephanie Haro, Andrea Megela Simmons, James A. Simmons

    Computational models of animal biosonar seek to identify critical aspects of echo processing responsible for the superior, real-time performance of echolocating bats and dolphins in target tracking and clutter rejection. The Spectrogram Correlation and Transformation (SCAT) model replicates aspects of biosonar imaging in both species by processing wideband biosonar sounds and echoes with auditory mechanisms identified from experiments with bats. The model acquires broadband biosonar broadcasts and echoes, represents them as time-frequency spectrograms using parallel bandpass filters, translates the filtered signals into ten parallel amplitude threshold levels, and then operates on the resulting time-of-occurrence values at each frequency to estimate overall echo range delay. It uses the structure of the echo spectrum by depicting it as a series of local frequency nulls arranged regularly along the frequency axis of the spectrograms after dechirping them relative to the broadcast. Computations take place entirely on the timing of threshold-crossing events for each echo relative to threshold-events for the broadcast. Threshold-crossing times take into account amplitude-latency trading, a physiological feature absent from conventional digital signal processing. Amplitude-latency trading transposes the profile of amplitudes across frequencies into a profile of time-registrations across frequencies. Target shape is extracted from the spacing of the object’s individual acoustic reflecting points, or glints, using the mutual interference pattern of peaks and nulls in the echo spectrum. These are merged with the overall range-delay estimate to produce a delay-based reconstruction of the object’s distance as well as its glints. Clutter echoes indiscriminately activate multiple parts in the null-detecting system, which then produces the equivalent glint-delay spacings in images, thus blurring the overall echo-delay estimates by adding spurious glint delays to the image. Blurring acts as an anticorrelation process that rejects clutter intrusion into perceptions.


  • Open Access

    Comparison of Two-Talker Attention Decoding from EEG with Nonlinear Neural Networks and Linear Methods

    by Gregory Ciccarelli, Michael Nolan, Joseph Perricone, Paul T. Calamia, Stephanie Haro, James O’sUllivan, Nima Mesgarani, Thomas F. Quatieri, Christopher J. Smalt

    Auditory attention decoding (AAD) through a brain-computer interface has had a flowering of developments since it was first introduced by Mesgarani and Chang (2012) using electrocorticograph recordings. AAD has been pursued for its potential application to hearing-aid design in which an attention-guided algorithm selects, from multiple competing acoustic sources, which should be enhanced for the listener and which should be suppressed. Traditionally, researchers have separated the AAD problem into two stages: reconstruction of a representation of the attended audio from neural signals, followed by determining the similarity between the candidate audio streams and the reconstruction. Here, we compare the traditional two-stage approach with a novel neural-network architecture that subsumes the explicit similarity step. We compare this new architecture against linear and non-linear (neural-network) baselines using both wet and dry electroencephalogram (EEG) systems. Our results indicate that the new architecture outperforms the baseline linear stimulus-reconstruction method, improving decoding accuracy from 66% to 81% using wet EEG and from 59% to 87% for dry EEG. Also of note was the finding that the dry EEG system can deliver comparable or even better results than the wet, despite the latter having one third as many EEG channels as the former. The 11-subject, wet-electrode AAD dataset for two competing, co-located talkers, the 11-subject, dry-electrode AAD dataset, and our software are available for further validation, experimentation, and modification.