Fabian Schmidt

Auditory speech comprehension is a multi-faceted process in which attention, prediction, and sensorimotor integration (via active sensing) interact with or complement each other. Although different conceptual models that focus on one of these aspects exist, we still lack a unified understanding of their role in speech processing. Here, we first replicated two recently published studies from our lab, confirming 1) a positive relationship between individual prediction tendencies and neural speech tracking, and 2) the phenomenon of ocular speech tracking - the tracking of attended speech by eye movements - and its shared contribution with neural activity to speech processing. In addition, we extended these findings with complementary analyses and investigated these phenomena in relation to each other in a multi-speaker paradigm with continuous, narrative speech. Importantly, prediction tendency and ocular speech tracking seem to be unrelated. In contrast to the shared contributions of oculomotor and neural activity to speech processing over a distributed set of brain regions that are critical for attention, individual prediction tendency and its relation to neural speech tracking seem to be largely independent of attention. Based on these findings, we propose a framework that aims to bridge the gaps between attention, prediction, and active (ocular) sensing in order to contribute to a holistic understanding of neural speech processing. In this speculative framework for listening, auditory inflow is, on a basic level, temporally modulated via active ocular sensing, and incoming information is interpreted based on probabilistic assumptions.

Tinnitus, the perception of sound without an external source, affects 10-15% of individuals, yet its neural mechanisms remain poorly understood. Building on evidence that chronological age predicts tinnitus risk beyond hearing loss, we tested the hypothesis that accelerated neural aging increases susceptibility to tinnitus by integrating cross-sectional resting-state MEG with longitudinal structural imaging data. In a large MEG dataset, spectral parametrization revealed that age-neural relationships were amplified in the tinnitus group compared to age-, sex- & hearing-matched controls. Complementing these findings, prospective analyses using data from the UK Biobank showed that among participants without tinnitus at baseline, stronger age-related declines in white-matter density predicted later tinnitus onset. Collectively, these converging functional and structural findings support accelerated brain aging as a key risk factor for tinnitus.

The power of electrophysiologically measured cortical activity decays with an approximately 1 /f^X function. The slope of this decay (i.e. the spectral exponent, X) is modulated by various factors such as age, cognitive states or psychiatric/neurological disorders. Interestingly, a mostly parallel line of research has also uncovered similar effects for the spectral slope in the electrocardiogram (ECG). This raises the question of whether these bodywide changes in spectral slopes are (in-)dependent. Focusing on well-established age-related changes in spectral slopes, we analyzed a total of 1282 recordings of magnetoencephalography (MEG) resting state measurements with concurrent ECG in an age-diverse sample (18–88 years). Using a diverse array of analytical approaches, we demonstrate that the aperiodic signal recorded via surface electrodes/sensors originates from multiple physiological sources. Furthermore, our results suggest that common ‘artifact’ rejection approaches (i.e. ICA) may not be sufficient to separate cardiac from neural activity. In particular, significant parts of age-related changes in aperiodic activity normally interpreted to be of neural origin can be explained by cardiac activity. Moreover, our results suggest that changes (flattening/steepening) of the spectral slope with age are dependent on the recording site and investigated frequency range. Our results highlight the complexity of aperiodic activity while raising concerns when interpreting aperiodic activity as ‘cortical’ without considering physiological influences.

Developmental dyslexia is marked by persistent deficits in reading and phonological awareness, potentially linked to atypical neural tracking of acoustic speech rhythms at frequencies below 10 Hz. Using magnetoencephalography, we compared adults with developmental dyslexia and matched controls while they listened to speech with varying levels of intelligibility (manipulated via noise vocoding). In controls, decreasing intelligibility led to systematic increases in the tracked frequency of the speech stream, replicating previous findings. This increase in tracking frequency was largely absent in individuals with developmental dyslexia. Unexpectedly, the magnitude of periodic speech tracking in the low frequency range was preserved in DD. Instead, the increase in central tracking frequency was linked to phonological awareness and reading performance.

The electric signals generated by physiological activity exhibit both activity patterns that are regularly repeating over time (i.e. periodic) and activity patterns that are temporally irregular (i.e. aperiodic). In recent years several algorithms have been proposed to separate the periodic from the aperiodic parts of the signal, such as the irregular-resampling auto-spectral analysis (IRASA, (Wen & Liu, 2016)). IRASA separates periodic and aperiodic components by up-/downsampling time domain signals and computing their respective auto-power spectra. Finally, the aperiodic component is isolated by averaging over the resampled auto-power spectra removing any frequency-specific activity. The aperiodic component can then be subtracted from the original power spectrum yielding the residual periodic component. PyRASA is a package that is built upon and extends the IRASA algorithm (Wen & Liu, 2016). The package not only allows its users to separate power spectra, but also contains functionality to further parametrize the periodic and aperiodic spectra, by means of peak detection and several slope fitting options (eg. spectral knees). Furthermore, we implemented a function to use the IRASA algorithm in the time-frequency domain allowing for a time-resolved spectral parametrization using IRASA.

The power of electrophysiologically measured cortical activity decays with an approximately 1/fX function. The slope of this decay (i.e. the spectral exponent, X) is modulated by various factors such as age, cognitive states or psychiatric/neurological disorders. Interestingly, a mostly parallel line of research has also uncovered similar effects for the spectral slope in the electrocardiogram (ECG). This raises the question whether these bodywide changes in spectral slopes are (in-)dependent. Focusing on well-established age-related changes in spectral slopes we analyzed a total of 1282 recordings of magnetoencephalography (MEG) resting state measurements with concurrent ECG in an age-diverse sample (18-88 years). Using a diverse array of analytical approaches, we demonstrate that the aperiodic signal recorded via surface electrodes/sensors originates from multiple physiological sources. Furthermore, our results suggest that common “artifact” rejection approaches (i.e. ICA) may not be sufficient to separate cardiac from neural activity. In particular, significant parts of age-related changes in aperiodic activity normally interpreted to be of neural origin can be explained by cardiac activity. Moreover, our results suggest that changes (flattening/steepening) of the spectral slope with age are dependent on the recording site and investigated frequency range. Our results highlight the complexity of aperiodic activity while raising concerns when interpreting aperiodic activity as “cortical“ without considering physiological influences.

Behavioral and neuroscientific studies have shown that watching a speaker's lip movements aids speech comprehension. Intriguingly, even when videos of speakers are presented silently, various cortical regions track auditory features, such as the envelope. Recently, we demonstrated that eye movements track low-level acoustic information when attentively listening to speech. In this study, we investigated whether ocular speech tracking occurs during visual speech and how it influences cortical silent speech tracking. Furthermore, we compared data from hearing individuals, congenitally deaf individuals, and those who became deaf or hard of hearing (DHH) later in life to assess how audiovisual listening experience and auditory deprivation (early vs late onset) affect neural and ocular speech tracking during silent lip-reading. Using magnetoencephalography (MEG), we examined ocular and neural speech tracking of 75 participants observing silent videos of a speaker played forward and backward. Our main finding is a clear ocular unheard speech tracking effect with dominance of <1 Hz, which was not present for lip movements. Similarly, we observed an ≤ 1.3 Hz effect of neural unheard speech tracking in temporal regions for hearing participants. Importantly, neural tracking was not directly linked to ocular tracking. Strikingly, across listening groups, deaf participants with auditory experience showed higher ocular speech tracking than hearing participants, while no ocular speech tracking effect was revealed for congenitally deaf participants in a very small sample. This study extends previous work by demonstrating the involvement of eye movements in speech processing, even in the absence of acoustic input.

Behavioral and neuroscientific studies have shown that watching a speaker’s lip movements aids speech comprehension. Intriguingly, even when videos of speakers are presented silently, various cortical regions track auditory features, such as the envelope. Recently, we demonstrated that eye movements track low-level acoustic information when attentively listening to speech. In this study we investigated whether ocular speech tracking occurs during visual speech and how it influences cortical silent speech tracking. Furthermore, we compared the data of hearing individuals with congenitally deaf individuals, and those with acquired deafness or hearing loss (DHH; Deaf or hard of hearing) to assess how auditory deprivation (early vs. late onset) affects neural and ocular speech tracking during silent lip-reading. Using magnetoencephalography (MEG), we examined ocular and neural speech tracking of 75 participants observing silent videos of a speaker played forward and backward. Our main finding is a clear ocular unheard speech tracking effect with a dominance <1 Hz, which was not present for the lip movements. Similarly, we observed a <=1.3 Hz effect of neural unheard speech tracking in temporal regions for hearing participants. Importantly, neural tracking was not directly linked to ocular tracking in this study. Strikingly, across different listening groups, deaf participants with auditory listening experience showed higher ocular speech tracking than hearing participants, while no ocular speech tracking effect was revealed for congenitally deaf participants in a very small sample. This study extends our previous work by demonstrating the involvement of eye movements in speech processing, even in the absence of acoustic input.

Over the last decades, cognitive neuroscience has identified a distributed set of brain regions that are critical for attention. Strong anatomical overlap with brain regions critical for oculomotor processes suggests a joint network for attention and eye movements. However, the role of this shared network in complex, naturalistic environments remains understudied. Here, we investigated eye movements in relation to (un)attended sentences of natural speech. Combining simultaneously recorded eye tracking and magnetoencephalographic data with temporal response functions, we show that gaze tracks attended speech, a phenomenon we termed ocular speech tracking. Ocular speech tracking even differentiates a target from a distractor in a multi-speaker context and is further related to intelligibility. Moreover, we provide evidence for its contribution to neural differences in speech processing, emphasizing the necessity to consider oculomotor activity in future research and in the interpretation of neural differences in auditory cognition.