Automated remote speech‐based testing of individuals with cognitive decline: Bayesian agreement of transcription accuracy
Authors:
Open Access
- 6 October 2024
- journal article
- research article
- Published by Wiley in Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring
- Vol. 16 (4) , e70011
Abstract
Introduction: We investigated the agreement between automated and gold‐standard manual transcriptions of telephone chatbot‐based semantic verbal fluency testing.Methods: We examined 78 cases from the Screening over Speech in Unselected Populations for Clinical Trials in AD (PROSPECT‐AD) study, including cognitively normal individuals and individuals with subjective cognitive decline, mild cognitive impairment, and dementia. We used Bayesian Bland–Altman analysis of word count and the qualitative features of semantic cluster size, cluster switches, and word frequencies.Results: We found high levels of agreement for word count, with a 93% probability of a newly observed difference being below the minimally important difference. The qualitative features had fair levels of agreement. Word count reached high levels of discrimination between cognitively impaired and unimpaired individuals, regardless of transcription mode.Discussion: Our results support the use of automated speech recognition particularly for the assessment of quantitative speech features, even when using data from telephone calls with cognitively impaired individuals in their homes.Highlights: High levels of agreement were found between automated and gold‐standard manual transcriptions of telephone chatbot‐based semantic verbal fluency testing, particularly for word count. The qualitative features had fair levels of agreement. Word count reached high levels of discrimination between cognitively impaired and unimpaired individuals, regardless of transcription mode. Automated speech recognition for the assessment of quantitative and qualitative speech features, even when using data from telephone calls with cognitively impaired individuals in their homes, seems feasible and reliable.Keywords
This publication has 41 references indexed in Scilit:
- The diagnosis of mild cognitive impairment due to Alzheimer's disease: Recommendations from the National Institute on Aging‐Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's diseaseAlzheimer's & Dementia, 2011
- The diagnosis of dementia due to Alzheimer's disease: Recommendations from the National Institute on Aging‐Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's diseaseAlzheimer's & Dementia, 2011
- Performance on a measure of category fluency in cognitively impaired elderlyAging, Neuropsychology, and Cognition, 2011
- Method agreement analysis: A review of correct methodologyTheriogenology, 2010
- Using verbal fluency to detect very mild dementia of the Alzheimer typeArchives of Clinical Neuropsychology, 2006
- Category and letter fluency in semantic dementia, primary progressive aphasia, and Alzheimer’s diseaseBrain and Language, 2006
- Interpretation of Changes in Health-related Quality of LifeMedical Care, 2003
- Clustering and switching on verbal fluency tests in Alzheimer's and Parkinson's diseaseJournal of the International Neuropsychological Society, 1998
- Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults.Neuropsychology, 1997
- STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENTThe Lancet, 1986