Galit Agmon

Psycholinguistic research and linguistic theory suggest that comprehending continuous speech involves segmenting it into sentences. Yet, direct neuroimaging evidence for this process remains scarce, partly because sentence boundaries coincide with acoustic, prosodic, lexical, and syntactic changes. Here, we used whole-brain fMRI to test whether sentence boundaries elicit a neural response beyond these accompanying speech properties during spontaneous speech comprehension. We modelled BOLD responses to sentence-final words alongside continuous regressors characterizing accompanying speech properties. This approach revealed sensitivity to continuously varying features of speech across multiple brain regions. Notably, the acoustic envelope predicted BOLD responses in bilateral primary auditory cortices. Responses associated with prosodic breaks and emphasis were found in right temporal cortex, syntactic bottom-up node count in the left superior temporal sulcus, and word concreteness in the parahippocampal gyri. Critically, sentence-final words were associated with an increased activation in right temporal regions, after accounting for acoustic, prosodic, lexical, semantic and syntactic properties. These findings provide evidence for a neural response to sentence boundaries consistent with active sentence segmentation. More generally, this study demonstrates the value of continuous-regressor modelling for investigating speech comprehension with fMRI.

The recent adoption of naturalistic paradigms in the psychology and neuroscience of language has necessitated methods for quantifying incremental syntactic processing complexity. However, there is little consensus about how complexity should be quantified. Existing studies employ a wide range of complexity metrics, reflecting divergent assumptions about the nature of grammatical representations and the mechanisms underlying syntactic processing. Because these assumptions are rarely made explicit, findings across studies cannot be reliably interpreted or consistently synthesized into general conclusions. The goal of the present study is to review the word-level syntactic complexity metrics that have been used to model behavioral and neural measures of syntactic processing. To organize the empirical literature, we introduce a taxonomy based on two complementary dimensions: the grammatical distinction between constituency and dependency representations, and the cognitive distinction between computation- and memory-related aspects of syntactic processing. Our review reveals substantial heterogeneity in how syntactic complexity is operationalized and interpreted, even among ostensibly similar metrics. To test this quantitatively, we applied hierarchical clustering to 26 metrics computed on a sample of 100 annotated sentences. Computation- and memory-related metrics largely clustered together, providing quantitative support for the relevance of our proposed taxonomy. However, this pattern emerged only when constituency- and dependency-based metrics were analyzed separately, showing that the choice of the grammatical formalism shapes how syntactic complexity is quantified. Taken together, these findings underscore the need for greater conceptual clarity in the use of syntactic complexity metrics and for more explicit linking hypotheses connecting linguistic structure to cognitive and neural processes. This review provides a useful framework for achieving these goals.

Spoken language is central to human communication, influencing cognition, learning, and social interactions. Despite its spontaneous nature, characterized by disfluencies, fillers, self-corrections and irregular syntax, it effectively serves its communicative purpose. Understanding how the brain processes natural language offers valuable insights into the neurobiology of language. Recent neuroscience advancements allow us to study neural processes in response to ongoing speech, requiring detailed, time-locked descriptions of speech material to capture the nuances of spoken language. While there are many speech-to-text tools available, obtaining a time-locked true verbatim transcript, reflecting everything that was uttered, requires additional effort to achieve an accurate representation. We demonstrate the challenges involved in the process of obtaining time-resolved annotation of spontaneous speech, by presenting two semi-automatic pipelines, developed for German and Hebrew but adaptable to other languages. The outputs of these pipelines enable analyses of the neural representation and processing of key linguistic features. We discuss the methodological challenges and opportunities posed by current state-of-the-art pipelines, and advocate for new lines of natural language processing research aimed at advancing our understanding of how the brain processes everyday language.

Purpose: Multiple methods have been suggested for quantifying syntactic complexity in speech. We compared eight automated syntactic complexity metrics to determine which best captured verified syntactic differences between old and young adults.Method: We used natural speech samples produced in a picture description task by younger (n= 76, ages 18–22 years) and older (n= 36, ages 53–89 years) healthy participants, manually transcribed and segmented into sentences. We manually verified that older participants produced fewer complex structures. We developed a metric of syntactic complexity using automatically extracted syntactic structures as features in a multidimensional metric. We compared our metric to seven other metrics: Yngve score, Frazier score, Frazier–Roark score, developmental level, syntactic frequency, mean dependency distance, and sentence length. We examined the success of each metric in identifying the age group using logistic regression models. We repeated the analysis with automatic transcription and segmentation using an automatic speech recognition (ASR) system.Results: Our multidimensional metric was successful in predicting age group (area under the curve [AUC] = 0.87), and it performed better than the other metrics. High AUCs were also achieved by the Yngve score (0.84) and sentence length (0.84). However, in a fully automated pipeline with ASR, the performance of these two metrics dropped (to 0.73 and 0.46, respectively), while the performance of the multidimensional metric remained relatively high (0.81).Conclusions: Syntactic complexity in spontaneous speech can be quantified by directly assessing syntactic structures and considering them in a multivariable manner. It can be derived automatically, saving considerable time and effort compared to manually analyzing large-scale corpora, while maintaining high face validity and robustness.Supplemental Material: https://doi.org/10.23641/asha.24964179

Clinical trials developing therapeutics for frontotemporal degeneration (FTD) focus on pathogenic variant carriers at preclinical stages. Objective, quantitative clinical assessment tools are needed to track stability and delayed disease onset. Natural speech can serve as an accessible, cost-effective assessment tool. We aimed to identify early changes in the natural speech of FTD pathogenic variant carriers before they become symptomatic. In this cohort study, speech samples of picture descriptions were collected longitudinally from healthy participants in observational studies at the University of Pennsylvania and Columbia University between 2007 and 2020. Participants were asymptomatic but at risk for familial FTD. Status as "carrier" or "noncarrier" was based on screening for known pathogenic variants in the participant's family. Thirty previously validated digital speech measures derived from automatic speech processing pipelines were selected a priori based on previous studies in patients with FTD and compared between asymptomatic carriers and noncarriers cross-sectionally and longitudinally. A total of 105 participants, all asymptomatic, included 41 carriers: 12 men [30%], mean age 43 ± 13 years; education, 16 ± 2 years; MMSE 29 ± 1; and 64 noncarriers: 27 men [42%]; mean age, 48 ± 14 years; education, 15 ± 3 years; MMSE 29 ± 1. We identified 4 speech measures that differed between carriers and noncarriers at baseline: mean speech segment duration (mean difference -0.28 seconds, 95% CI -0.55 to -0.02, p = 0.04); word frequency (mean difference 0.07, 95% CI 0.008-0.14, p = 0.03); word ambiguity (mean difference 0.02, 95% CI 0.0008-0.05, p = 0.04); and interjection count per 100 words (mean difference 0.33, 95% CI 0.07-0.59, p = 0.01). Three speech measures deteriorated over time in carriers only: particle count per 100 words per month (β = -0.02, 95% CI -0.03 to -0.004, p = 0.009); total narrative production time in seconds per month (β = -0.24, 95% CI -0.37 to -0.12, p < 0.001); and total number of words per month (β = -0.48, 95% CI -0.78 to -0.19, p = 0.002) including in 3 carriers who later converted to symptomatic disease. Using automatic processing pipelines, we identified early changes in the natural speech of FTD pathogenic variant carriers in the presymptomatic stage. These findings highlight the potential utility of natural speech as a digital clinical outcome assessment tool in FTD, where objective and quantifiable measures for abnormal behavior and language are lacking.

Spontaneous real-life speech is imperfect in many ways. It contains disfluencies and ill-formed utterances and has a highly variable rate. When listening to spontaneous speech, the brain needs to contend with these features in order to extract the speaker’s meaning. Here, we studied how the neural response is affected by four specific factors that are prevalent in spontaneous colloquial speech: (1) the presence of fillers, (2) the need to detect syntactic boundaries in disfluent speech, and (3) variability in speech rate. Neural activity was recorded (using electroencephalography) from individuals as they listened to an unscripted, spontaneous narrative, which was analyzed in a time-resolved fashion to identify fillers and detect syntactic boundaries. When considering these factors in a speech-tracking analysis, which estimates a temporal response function (TRF) to describe the relationship between the stimulus and the neural response it generates, we found that the TRF was affected by all of them. This response was observed for lexical words but not for fillers, and it had an earlier onset for opening words vs. closing words of a clause and for clauses with slower speech rates. These findings broaden ongoing efforts to understand neural processing of speech under increasingly realistic conditions. They highlight the importance of considering the imperfect nature of real-life spoken language, linking past research on linguistically well-formed and meticulously controlled speech to the type of speech that the brain actually deals with on a daily basis.

Negated sentences are known to be more cognitively taxing than positive ones (i.e., polarity effect). We present evidence that two factors contribute to the polarity effect in verification tasks: processing the sentence and verifying its truth value. To quantify the relative contribution of each, we used a delayed verification task. The results show that even when participants are given a considerable amount of time for processing the sentence prior to verification, the polarity effect is not entirely eliminated. We suggest that this sustained effect stems from a retained negation-containing representation in working memory. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

When faced with situations where many people talk at once, individuals can employ different listening strategies to deal with the cacophony of speech sounds and to achieve different goals. In this fMRI study, we investigated how the pattern of neural activity is affected by the type of attention applied to speech in a simulated “cocktail party.” Specifically, we compared brain activation patterns when listeners “attended selectively” to only one speaker and ignored all others, versus when they “distributed their attention” and followed several concurrent speakers. Conjunction analysis revealed a highly overlapping network of regions activated for both types of attention, including auditory association cortex (bilateral STG/STS) and frontoparietal regions related to speech processing and attention (bilateral IFG/insula, right MFG, left IPS). Activity within nodes of this network, though, was modulated by the type of attention required as well as the number of competing speakers. Auditory and speech-processing regions exhibited higher activity during distributed attention, whereas frontoparietal regions were activated more strongly during selective attention. These results suggest a common “attention to speech” network, which provides the computational infrastructure to deal effectively with multi-speaker input, but with sufficient flexibility to implement different prioritization strategies and to adapt to different listener goals.

The processing of sentences with negative quantifiers (e.g., few) is more costly than of sentences that contain their positive counterparts (e.g., many). While this polarity effect is robust and reliably replicable, its neurological bases are not well understood. In this study, we use functional magnetic resonance imaging (fMRI) paradigm for 30 participants to assess the polarity effect in sentences with polar quantifiers, and compare it with the polarity effect of polar adjectives. Both in quantifiers and in adjectives, the polarity effect manifests in the anterior insula bilaterally. The polarity effect in quantifiers, however, shows greater activation in the left hemisphere than it does for adjectives. In particular, left inferior frontal gyrus (IFG) and left superior temporal sulcus (STS) show increased activation for polarity in quantifiers than in adjectives, which is the evidence for the specific involvement of the language network in this type of polarity processing. Using the polarity effect in adjectives as a control, we provide further evidence for the linguistic complexity that negative quantifiers implicate on processing.