The CloudUPDRS smartphone software in Parkinson’s study: cross-validation against blinded human raters
Open Access
- 8 December 2020
- journal article
- research article
- Published by Springer Nature in npj Parkinson's Disease
- Vol. 6 (1) , 1-8
Abstract
Digital assessments of motor severity could improve the sensitivity of clinical trials and personalise treatment in Parkinson’s disease (PD) but have yet to be widely adopted. Their ability to capture individual change across the heterogeneous motor presentations typical of PD remains inadequately tested against current clinical reference standards. We conducted a prospective, dual-site, crossover-randomised study to determine the ability of a 16-item smartphone-based assessment (the index test) to predict subitems from the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale part III (MDS-UPDRS III) as assessed by three blinded clinical raters (the reference-standard). We analysed data from 60 subjects (990 smartphone tests, 2628 blinded video MDS-UPDRS III subitem ratings). Subject-level predictive performance was quantified as the leave-one-subject-out cross-validation (LOSO-CV) accuracy. A pre-specified analysis classified 70.3% (SEM 5.9%) of subjects into a similar category to any of three blinded clinical raters and was better than random (36.7%; SEM 4.3%) classification. Post hoc optimisation of classifier and feature selection improved performance further (78.7%, SEM 5.1%), although individual subtests were variable (range 53.2–97.0%). Smartphone-based measures of motor severity have predictive value at the subject level. Future studies should similarly mitigate against subjective and feature selection biases and assess performance across a range of motor features as part of a broader strategy to avoid overly optimistic performance estimates.Keywords
This publication has 26 references indexed in Scilit:
- Developing a Tool for Remote Digital Assessment of Parkinson's DiseaseMovement Disorders Clinical Practice, 2015
- Detecting and monitoring the symptoms of Parkinson's disease using smartphones: A pilot studyParkinsonism & Related Disorders, 2015
- The ongoing pursuit of neuroprotective therapies in Parkinson diseaseNature Reviews Neurology, 2014
- Smartphone-based evaluation of parkinsonian hand tremor: Quantitative measurements vs clinical assessment scores2014
- Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sampleNeurology, 2011
- Movement Disorder Society‐sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS‐UPDRS): Scale presentation and clinimetric testing resultsMovement Disorders, 2008
- The relevance of the Lewy body to the pathogenesis of idiopathic Parkinson's disease.Journal of Neurology, Neurosurgery & Psychiatry, 1988
- Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluationClinical Psychology Review, 1988
- Measuring nominal scale agreement among many raters.Psychological Bulletin, 1971
- ParkinsonismNeurology, 1967