Evaluation of the Predictive Ability and User Acceptance of Panoramix 2.0, an AI-Based E-Health Tool for the Detection of Cognitive Impairment
Open Access
- 22 October 2022
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 11 (21) , 3424
Abstract
The high prevalence of Alzheimer-type dementia and the limitations of traditional neuropsychological tests motivate the introduction of new cognitive assessment methods. We discuss the validation of an all-digital, ecological and non-intrusive e-health application for the early detection of cognitive impairment, based on artificial intelligence for patient classification, and more specifically on machine learning algorithms. To evaluate the discrimination power of this application, a cross-sectional pilot study was carried out involving 30 subjects: 10 health control subjects (mean age: 75.62 years); 14 individuals with mild cognitive impairment (mean age: 81.24 years) and 6 early-stage Alzheimer’s patients (mean age: 80.44 years). The study was carried out in two separate sessions in November 2021 and January 2022. All participants completed the study, and no concerns were raised about the acceptability of the test. Analysis including socio-demographics and game data supports the prediction of participants’ cognitive status using machine learning algorithms. According to the performance metrics computed, best classification results are obtained a Multilayer Perceptron classifier, Support Vector Machines and Random Forest, respectively, with weighted recall values >= 0.9784 ± 0.0265 and F1-score = 0.9764 ± 0.0291. Furthermore, thanks to hyper-parameter optimization, false negative rates were dramatically reduced. Shapley’s additive planning (SHAP) applied according to the eXplicable AI (XAI) method, made it possible to visually and quantitatively evaluate the importance of the different features in the final classification. This is a relevant step ahead towards the use of machine learning and gamification to early detect cognitive impairment. In addition, this tool was designed to support self-administration, which could be a relevant aspect in confinement situations with limited access to health professionals. However, further research is required to identify patterns that may help to predict or estimate future cognitive damage and normative data.Keywords
Funding Information
- European Union’s Horizon 2020 Research and Innovation Program (857223, PID2020-115137RB-I00)
- Spanish Ministry of Science and Innovation (857223, PID2020-115137RB-I00)
This publication has 45 references indexed in Scilit:
- Verbal Serial List Learning in Mild Cognitive Impairment: A Profile Analysis of Interference, Forgetting, and ErrorsJournal of the International Neuropsychological Society, 2011
- Data mining methods in the prediction of Dementia: A real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forestsBMC Research Notes, 2011
- Alzheimer’s Disease: The Challenge of the Second CenturyScience Translational Medicine, 2011
- Evaluation of Spatial Processing in Virtual Reality Using Functional Magnetic Resonance Imaging (fMRI)Cyberpsychology, Behavior, and Social Networking, 2010
- A six stage approach for the diagnosis of the Alzheimer’s disease based on fMRI dataJournal of Biomedical Informatics, 2010
- Use of Virtual Reality Tasks to Assess Prospective Memory: Applicability and EvidenceBrain Impairment, 2009
- Use of the Virtual Action Planning Supermarket for the Diagnosis of Mild Cognitive ImpairmentDementia and Geriatric Cognitive Disorders, 2009
- Application and comparison of classification algorithms for recognition of Alzheimer's disease in electrical brain activity (EEG)Journal of Neuroscience Methods, 2007
- Mild cognitive impairment as a diagnostic entityJournal of Internal Medicine, 2004
- Different error pattern of episodic and semantic memory in Alzheimer's disease and Parkinson's disease with dementiaNeuropsychologia, 1989