Nathan Weisz

Auditory speech comprehension is a multi-faceted process in which attention, prediction, and sensorimotor integration (via active sensing) interact with or complement each other. Although different conceptual models that focus on one of these aspects exist, we still lack a unified understanding of their role in speech processing. Here, we first replicated two recently published studies from our lab, confirming 1) a positive relationship between individual prediction tendencies and neural speech tracking, and 2) the phenomenon of ocular speech tracking - the tracking of attended speech by eye movements - and its shared contribution with neural activity to speech processing. In addition, we extended these findings with complementary analyses and investigated these phenomena in relation to each other in a multi-speaker paradigm with continuous, narrative speech. Importantly, prediction tendency and ocular speech tracking seem to be unrelated. In contrast to the shared contributions of oculomotor and neural activity to speech processing over a distributed set of brain regions that are critical for attention, individual prediction tendency and its relation to neural speech tracking seem to be largely independent of attention. Based on these findings, we propose a framework that aims to bridge the gaps between attention, prediction, and active (ocular) sensing in order to contribute to a holistic understanding of neural speech processing. In this speculative framework for listening, auditory inflow is, on a basic level, temporally modulated via active ocular sensing, and incoming information is interpreted based on probabilistic assumptions.

Predictive processing accounts hold that the brain continuously generates predictions and updates internal models from error signals, but it remains unclear whether prediction representation and model-updating reflect a single graded computation or computationally distinct operations implemented in dissociable systems. We recorded magnetoencephalography (MEG) and pupillometry while participants listened to continuous natural speech, quantifying word-by-word lexical surprise and semantic prediction error with a GPT-2 model. Broadly speaking, lexical surprise was tracked in a predominantly graded fashion, whereas the response to semantic error was better captured by a rectified linear (ReLU) gating function - i.e., that responds only when the error exceeds a recent contextual baseline. There were also some interesting dissociations in the strength of these relative effects. The auditory cortex tracked lexical surprise in a graded, continuous manner, with later cortical stages reflecting semantic error. By contrast, tonic pupil diameter and source-localised brainstem responses were predominantly captured by a gated response to semantic error. A pupil-coupling analysis confirmed that this gating signature was statistically shared between pupil-linked arousal and brainstem-localised, but not cortical, responses. Together, these findings reveal a division of labour in which the cortex maintains a continuous, high-fidelity map of predictive information while a pupil-linked model-update system acts as a selective gate, engaged specifically by meaning-disrupting events that have crossed a threshold level of error. This asymmetry suggests that signals serving continuous parsing of prediction error and signals serving model revision may be computationally dissociated, with a range of implications for our understanding of the intrinsic interactions between mechanisms serving perception and learning.

Tinnitus, the perception of sound without an external source, affects 10-15% of individuals, yet its neural mechanisms remain poorly understood. Building on evidence that chronological age predicts tinnitus risk beyond hearing loss, we tested the hypothesis that accelerated neural aging increases susceptibility to tinnitus by integrating cross-sectional resting-state MEG with longitudinal structural imaging data. In a large MEG dataset, spectral parametrization revealed that age-neural relationships were amplified in the tinnitus group compared to age-, sex- & hearing-matched controls. Complementing these findings, prospective analyses using data from the UK Biobank showed that among participants without tinnitus at baseline, stronger age-related declines in white-matter density predicted later tinnitus onset. Collectively, these converging functional and structural findings support accelerated brain aging as a key risk factor for tinnitus.

While eye movements have been shown to track the speech envelope, it is unknown whether this reflects a hard-wired mechanism or one shaped by (lifetime) audiovisual experience. Further, questions remain about whether ocular tracking is modulated by speech intelligibility and which brain regions drive these synchronized eye movements. Here, we investigate ocular speech tracking in blindfolded early blind, late blind, and sighted individuals using magnetoencephalography (MEG) and source-reconstructed oculomotor signals while participants listened to narrative speech of varying intelligibility. We found that oculomotor activity tracks acoustic speech features and, unlike neural speech tracking, is not modulated by intelligibility. Interestingly, we found effects reflected in two frequency-specific components: a low-frequency (∼1 Hz) effect present across all groups, indicating that visual experience is not required, and a high-frequency (∼6 Hz) effect reduced in early- and late-blind individuals. Moreover, this finding is not driven by cerebro-ocular connectivity, as late-blind individuals exhibit stronger connectivity between the eyes and the left temporal cortices without a corresponding increase in ocular tracking. In conclusion, ocular speech tracking seems to respond selectively to acoustic but not to intelligibility features of speech, and it does not require visual experience to develop. It may thus represent a hard-wired oculomotor mechanism within the oculo-cerebral network involved in speech processing. Significance Statement: Eye movements provide a unique window into the interaction between auditory and visual systems. By studying early blind, late blind, and sighted individuals, we demonstrate that speech-related eye movements arise from at least two distinct mechanisms: a low-frequency component that occurs independently of (lifetime) visual experience and is linked to processing of acoustic speech features, and a high-frequency component shaped by prior visual exposure. Importantly, speech intelligibility - unlike its impact on neural measures - does not modulate these ocular responses. This dissociation suggests that eye movements reflect mechanisms of spoken language processing that are independent of intelligibility, thereby revealing novel pathways of auditory-motor coupling and broadening our understanding of sensory integration in the absence of vision.

The power of electrophysiologically measured cortical activity decays with an approximately 1 /f^X function. The slope of this decay (i.e. the spectral exponent, X) is modulated by various factors such as age, cognitive states or psychiatric/neurological disorders. Interestingly, a mostly parallel line of research has also uncovered similar effects for the spectral slope in the electrocardiogram (ECG). This raises the question of whether these bodywide changes in spectral slopes are (in-)dependent. Focusing on well-established age-related changes in spectral slopes, we analyzed a total of 1282 recordings of magnetoencephalography (MEG) resting state measurements with concurrent ECG in an age-diverse sample (18–88 years). Using a diverse array of analytical approaches, we demonstrate that the aperiodic signal recorded via surface electrodes/sensors originates from multiple physiological sources. Furthermore, our results suggest that common ‘artifact’ rejection approaches (i.e. ICA) may not be sufficient to separate cardiac from neural activity. In particular, significant parts of age-related changes in aperiodic activity normally interpreted to be of neural origin can be explained by cardiac activity. Moreover, our results suggest that changes (flattening/steepening) of the spectral slope with age are dependent on the recording site and investigated frequency range. Our results highlight the complexity of aperiodic activity while raising concerns when interpreting aperiodic activity as ‘cortical’ without considering physiological influences.

Developmental dyslexia is marked by persistent deficits in reading and phonological awareness, potentially linked to atypical neural tracking of acoustic speech rhythms at frequencies below 10 Hz. Using magnetoencephalography, we compared adults with developmental dyslexia and matched controls while they listened to speech with varying levels of intelligibility (manipulated via noise vocoding). In controls, decreasing intelligibility led to systematic increases in the tracked frequency of the speech stream, replicating previous findings. This increase in tracking frequency was largely absent in individuals with developmental dyslexia. Unexpectedly, the magnitude of periodic speech tracking in the low frequency range was preserved in DD. Instead, the increase in central tracking frequency was linked to phonological awareness and reading performance.

How does the brain integrate or segregate multiple sensory inputs? In our study, we used magnetoencephalography to comprehensively investigate local and network dynamics before and after the presentation of sequences of beeps and flashes that are sometimes perceived as synchronous or asynchronous. Frequency-tagging and spectral analysis revealed that under identical physical conditions, subjective perceptual fusion of multisensory signals elicits stronger evoked sensory responses in both visual and auditory cortices. Crucially, perceptual fusion is preceded by locally reduced pre-stimulus alpha activity, as well as enhanced functional connectivity between auditory and visual cortices within the pre-stimulus gamma activity. We also show that pre-stimulus neuronal states in the occipital cortex predict post-stimulus sensory responses to auditory sequences in the temporal cortex, but not the opposite. The current findings provide evidence that the perceptual fusion of multisensory events relate to a complex interplay between local and inter-sensory neuronal states before and after stimulus presentation. Significance Statement How does the brain blend information from multiple sensory systems into one holistic perception? We developed a novel frequency-tagging paradigm in Magnetoencephalography (MEG) to robustly estimate the sensory responses to a series of flashes and beeps in early sensory areas, and to demonstrate how these responses are shaped by ongoing fluctuations in local, and long-range network activity. By combining the steady-state method with spectral, and functional connectivity analysis of ongoing oscillations in source space, the current study provides novel insights into the relationship between internal brain states and the way we perceive multisensory events.

The electric signals generated by physiological activity exhibit both activity patterns that are regularly repeating over time (i.e. periodic) and activity patterns that are temporally irregular (i.e. aperiodic). In recent years several algorithms have been proposed to separate the periodic from the aperiodic parts of the signal, such as the irregular-resampling auto-spectral analysis (IRASA, (Wen & Liu, 2016)). IRASA separates periodic and aperiodic components by up-/downsampling time domain signals and computing their respective auto-power spectra. Finally, the aperiodic component is isolated by averaging over the resampled auto-power spectra removing any frequency-specific activity. The aperiodic component can then be subtracted from the original power spectrum yielding the residual periodic component. PyRASA is a package that is built upon and extends the IRASA algorithm (Wen & Liu, 2016). The package not only allows its users to separate power spectra, but also contains functionality to further parametrize the periodic and aperiodic spectra, by means of peak detection and several slope fitting options (eg. spectral knees). Furthermore, we implemented a function to use the IRASA algorithm in the time-frequency domain allowing for a time-resolved spectral parametrization using IRASA.

The power of electrophysiologically measured cortical activity decays with an approximately 1/fX function. The slope of this decay (i.e. the spectral exponent, X) is modulated by various factors such as age, cognitive states or psychiatric/neurological disorders. Interestingly, a mostly parallel line of research has also uncovered similar effects for the spectral slope in the electrocardiogram (ECG). This raises the question whether these bodywide changes in spectral slopes are (in-)dependent. Focusing on well-established age-related changes in spectral slopes we analyzed a total of 1282 recordings of magnetoencephalography (MEG) resting state measurements with concurrent ECG in an age-diverse sample (18-88 years). Using a diverse array of analytical approaches, we demonstrate that the aperiodic signal recorded via surface electrodes/sensors originates from multiple physiological sources. Furthermore, our results suggest that common “artifact” rejection approaches (i.e. ICA) may not be sufficient to separate cardiac from neural activity. In particular, significant parts of age-related changes in aperiodic activity normally interpreted to be of neural origin can be explained by cardiac activity. Moreover, our results suggest that changes (flattening/steepening) of the spectral slope with age are dependent on the recording site and investigated frequency range. Our results highlight the complexity of aperiodic activity while raising concerns when interpreting aperiodic activity as “cortical“ without considering physiological influences.