Dementia and MCI Detection Based on Comprehensive Facial Expression Analysis From Videos During Conversation
- 7 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (5) , 3537-3548
Abstract
The development of a cost-effective digital biomarker for detecting dementia is highly needed. While numerous studies have explored dementia detection through speech and natural language analysis, only a few studies have focused on dementia detection using face video recordings, and more in-depth research is needed. In this paper, we propose a method for detecting dementia and mild cognitive impairment (MCI), a pre-dementia stage, by utilizing four types of facial expression features extracted from recorded videos of participants. These features include Action Units, emotion categories, Valence-Arousal, and face embeddings. From the above features obtained from each video frame, various statistical information was extracted and used as features, and predictions were performed using a decision tree-based model. Our method was evaluated using face video recordings during conversations. The method achieved an area under the receiver operating characteristic curve (AUC) of 0.933 for dementia detection and 0.889 for MCI detection. Statistical analysis of facial expression features revealed that participants with dementia had fewer positive emotions, more negative emotions, and lower valence and arousal than healthy participants. These results indicate that the proposed method could serve as an explainable screening tool for the early detection of dementia and MCI.Keywords
Funding Information
- JST ASPIRE (JPMJAP2326)
This publication has 49 references indexed in Scilit:
- Reading Between the Frames: Multi-modal Depression Detection in Videos from Non-verbal Cues2024
- MC-ViViT: Multi-branch Classifier-ViViT to detect Mild Cognitive Impairment in older adults using facial videosExpert Systems with Applications, 2023
- 2023 Alzheimer's disease facts and figuresAlzheimer's & Dementia, 2023
- Pre-training Strategies and Datasets for Facial Representation Learning2022
- The project for objective measures using computational psychiatry technology (PROMPT): Rationale, design, and methodologyContemporary Clinical Trials Communications, 2020
- AFEW-VA database for valence and arousal estimation in-the-wildImage and Vision Computing, 2017
- Increased subjective experience of non-target emotions in patients with frontotemporal dementia and Alzheimer’s diseaseCurrent Opinion in Behavioral Sciences, 2017
- BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression databaseImage and Vision Computing, 2014
- Clinical and cost effectiveness of services for early diagnosis and intervention in dementiaInternational Journal of Geriatric Psychiatry, 2009
- “Mini-mental state”Journal of Psychiatric Research, 1975