Benjamin Glenn Schultz

Neurodegenerative disease often affects speech. Speech acoustics can be used as objective clinical markers of pathology. Previous investigations of pathological speech have primarily compared controls with one specific condition and excluded comorbidities. We broaden the utility of speech markers by examining how multiple acoustic features can delineate diseases. We used supervised machine learning with gradient boosting (CatBoost) to delineate healthy speech from speech of people with multiple sclerosis or Friedreich ataxia. Participants performed a diadochokinetic task where they repeated alternating syllables. We subjected 74 spectral and temporal prosodic features from the speech recordings to machine learning. Results showed that Friedreich ataxia, multiple sclerosis and healthy controls were all identified with high accuracy (over 82%). Twenty-one acoustic features were strong markers of neurodegenerative diseases, falling under the categories of spectral qualia, spectral power, and speech rate. We demonstrated that speech markers can delineate neurodegenerative diseases and distinguish healthy speech from pathological speech with high accuracy. Findings emphasize the importance of examining speech outcomes when assessing indicators of neurodegenerative disease. We propose large-scale initiatives to broaden the scope for differentiating other neurological diseases and affective disorders.

Background: Digital biomarkers continue to make headway in the clinic and clinical trials for neurological conditions. Speech is a domain with great promise. Objectives: Using Friedreich ataxia (FRDA) as an exemplar population, we aimed to align objective measures of speech with markers of disease severity, speech related quality of life and subjective judgements of speech using supervised machine learning techniques. Methods: 132 participants with genetically confirmed diagnosis of FRDA were assessed using digital speech tests, disease severity scores (Friedreich Ataxia Rating Scale, FARS) and speech related quality of life ratings over a 10-year period. Speech was analyzed perceptually by expert listeners for intelligibility (ability to be understood) and naturalness (deviance from healthy norm) and acoustically across 344 features. Features were selected and presented into a random forest and a support vector machine classifier in a standard supervised learning setup designed to replicate expert-produced scores. Results: We demonstrate a subset of measures are strongly associated with all four clinical scales. Objective speech data replicated experts’ assessments of naturalness and intelligibility. These scores represent a lower level of variability than observed between subjective listener ratings. Findings provide evidence there are specific objective markers of speech that change over time and reflect clinical aspects of the disease. Discussion: The use of a large dataset yielded a speech assay capable of accurately approximating expert listener ratings of key clinical aspects of dysarthria severity. Distinct but complementary subsets align with disease severity and speech related quality of life.

Purpose: The human voice changes with the progression of neurological disease and the onset of diseases that affect articulators, often decreasing the effectiveness of communication. These changes can be objectively measured using signal processing techniques that extract acoustic features. When measuring acoustic features, there are often several steps and assumptions that might be known to experts in acoustics and phonetics, but are less transparent for other disciplines (e.g., clinical medicine, speech pathology, engineering, and data science). This tutorial describes these signal processing techniques, explicitly outlines the underlying steps for accurate measurement, and discusses the implications of clinical acoustic markers. Conclusions: We establish a vocabulary using straightforward terms, provide visualizations to achieve common ground, and guide understanding for those outside the domains of acoustics and auditory signal processing. Where possible, we highlight the best practices for measuring clinical acoustic markers and suggest resources for obtaining and further understanding these measures.

Neurodegenerative diseases often affect speech. Speech acoustics can be used as objective clinical markers of pathology. Previous investigations of pathological speech have primarily compared controls with one specific condition and excluded comorbidities. We broaden the utility of speech markers by examining how multiple acoustic features can delineate diseases. We used supervised machine learning with gradient boosting (CatBoost) to differentiate healthy speech and speech from people with multiple sclerosis or Friedreich ataxia. Participants performed a diadochokinetic task where they repeated alternating syllables. We extracted 74 spectral and temporal prosodic features from the speech recordings, which were subjected to machine learning. Results showed that Friedreich ataxia, multiple sclerosis and healthy controls were all identified with high accuracy (over 82%). Twenty-one acoustic features were strong markers of neurodegenerative diseases, falling under the categories of spectral qualia, spectral power, and speech rate. We demonstrated that speech markers can delineate neurodegenerative diseases and distinguish healthy speech from pathological speech with high accuracy. Findings emphasize the importance of examining speech outcomes when assessing indicators of neurodegenerative disease. We propose large-scale initiatives to broaden the scope for differentiating other neurological diseases and affective disorders.

Automatic speech recognition (ASR) could potentially improve communication by providing transcriptions of speech in real time. ASR is particularly useful for people with progressive disorders that lead to reduced speech intelligibility or difficulties performing motor tasks. ASR services are usually trained on healthy speech and may not be optimized for impaired speech, creating a barrier for accessing augmented assistance devices. We tested the performance of three state-of-the-art ASR platforms on two groups of people with neurodegenerative disease and healthy controls. We further examined individual differences that may explain errors in ASR services within groups, such as age and sex. Speakers were recorded while reading a standard text. Speech was elicited from individuals with multiple sclerosis, Friedreich’s ataxia, and healthy controls. Recordings were manually transcribed and compared to ASR transcriptions using Amazon Web Services, Google Cloud, and IBM Watson. Accuracy was measured as the proportion of words that were correctly classified. ASR accuracy was higher for controls than clinical groups, and higher for multiple sclerosis compared to Friedreich’s ataxia for all ASR services. Amazon Web Services and Google Cloud yielded higher accuracy than IBM Watson. ASR accuracy decreased with increased disease duration. Age and sex did not significantly affect ASR accuracy. ASR faces challenges for people with neuromuscular disorders. Until improvements are made in recognizing less intelligible speech, the true value of ASR for people requiring augmented assistance devices and alternative communication remains unrealized. We suggest potential methods to improve ASR for those with impaired speech.

Wearing face masks (alongside physical distancing) provides some protection against infection from COVID-19. Face masks can also change how we communicate and subsequently affect speech signal quality. Here we investigated how three face mask types (N95, surgical and cloth) affect acoustic analysis of speech and perceived intelligibility in healthy subjects. We compared speech produced with and without the different masks on acoustic measures of timing, frequency, perturbation and power spectral density. Speech clarity was also examined using a standardized intelligibility tool by blinded raters. Mask type impacted the power distribution in frequencies above 3kHz for both the N95 and surgical masks. Measures of timing and spectral tilt also differed across mask conditions. Cepstral and harmonics to noise ratios remained flat across mask type. No differences were observed across conditions for word or sentence intelligibility measures. Our data show that face masks change the speech signal, but some specific acoustic features remain largely unaffected (e.g., measures of voice quality) irrespective of mask type. Outcomes have bearing on how future speech studies are run when personal protective equipment is worn.

Most human communication is carried by modulations of the voice. However, a wide range of cultures has developed alternative forms of communication that make use of a whistled sound source. For example, whistling is used as a highly salient signal for capturing attention, and can have iconic cultural meanings such as the catcall, enact a formal code as in boatswain's calls or stand as a proxy for speech in whistled languages. We used real-time magnetic resonance imaging to examine the muscular control of whistling to describe a strong association between the shape of the tongue and the whistled frequency. This bioacoustic profile parallels the use of the tongue in vowel production. This is consistent with the role of whistled languages as proxies for spoken languages, in which one of the acoustical features of speech sounds is substituted with a frequency-modulated whistle. Furthermore, previous evidence that non-human apes may be capable of learning to whistle from humans suggests that these animals may have similar sensorimotor abilities to those that are used to support speech in humans.

Auditory feedback of actions provides additional information about the timing of one’s own actions and those of others. However, little is known about how musicians and nonmusicians integrate auditory feedback from multiple sources to regulate their own timing or to (intentionally or unintentionally) coordinate with a partner. We examined how musical expertise modulates the role of auditory feedback in a two-person synchronization–continuation tapping task. Pairs of individuals were instructed to tap at a rate indicated by an initial metronome cue in all four auditory feedback conditions: no feedback, self-feedback (cannot hear their partner), other feedback (cannot hear themselves), or full feedback (both self and other). Participants within a pair were either both musically trained (musicians), both untrained (nonmusicians), or one musically trained and one untrained (mixed). Results demonstrated that all three pair types spontaneously synchronized with their partner when receiving other or full feedback. Moreover, all pair types were better at maintaining the metronome rate with self-feedback than with no feedback. Musician pairs better maintained the metronome rate when receiving other feedback than when receiving no feedback; in contrast, nonmusician pairs were worse when receiving other or full feedback compared to no feedback. Both members of mixed pairs maintained the metronome rate better in the other and full feedback conditions than in the no feedback condition, similar to musician pairs. Overall, nonmusicians benefited from musicians’ expertise without negatively influencing musicians’ ability to maintain the tapping rate. One implication is that nonmusicians may improve their beat-keeping abilities by performing tasks with musically skilled individuals.