Beyond the Clock: Exploring Multimodal Behavior Markers of Mild Cognitive Impairment in Older Adults during Clock Drawing Test
Open Access
- 2 December 2025
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 9 (4) , 1-30
Abstract
Early detection of Mild Cognitive Impairment (MCI) offers a critical window for intervention before progression to dementia. This study presents a novel approach to MCI screening through multimodal behavioral analysis during the Clock Drawing Test (CDT), implementing a comprehensive data collection system using standard tablet devices. Our approach analyzes three behavioral dimensions: drawing trajectories, facial expressions, and hand motion patterns. We developed an integrated framework combining Convolutional Neural Networks with multi-head attention mechanisms for drawing analysis, alongside specialized feature extraction methods for facial and motion data. To effectively integrate these behavioral markers, we proposed a Higher-order Canonical Correlation Analysis (HCCA) algorithm that models complex relationships between modalities. In our primary study with 69 MCI participants and 133 age-matched controls, we identified distinct behavioral patterns: temporal drawing anomalies, altered facial behaviors including increased gaze variability, and distinctive motion patterns reflecting changes in movement stability. Our multimodal approach achieved an AUC of 0.92 (accuracy: 92.68%, sensitivity: 90.91%), with HCCA further improving performance to an AUC of 0.97. External validation with 76 additional participants demonstrated good generalizability (AUC: 0.86). These findings reveal how cognitive impairment manifests across multiple behavioral channels offer clinicians precise, interpretable quantification of cognitive function for more accurate MCI screening.Keywords
Funding Information
- National Science and Technology Innovation 2030, Noncommunicable Chronic Diseases-National Science and Technology Major Project (2024ZD0524300, 2024ZD0524302)
This publication has 48 references indexed in Scilit:
- Wearable sensors for monitoring caregivers of people with dementia: a scoping reviewEuropean Geriatric Medicine, 2024
- Education, neighborhood environment, and cognitive decline: Findings from two prospective cohort studies of older adults in ChinaAlzheimer's & Dementia, 2022
- A Novel deep neural network-based emotion analysis system for automatic detection of mild cognitive impairment in the elderlyNeurocomputing, 2022
- Effects of low-pass filter combinations on lower extremity joint moments in distance runningJournal of Biomechanics, 2019
- Touchscreen games to detect cognitive impairment in senior adults. A user-interaction pilot studyInternational Journal of Medical Informatics, 2019
- Handwriting analysis to support neurodegenerative diseases diagnosis: A reviewPattern Recognition Letters, 2019
- Comparison of Computerized and Paper-and-Pencil Memory Tests in Detection of Mild Cognitive Impairment and Dementia: A Systematic Review and Meta-analysis of Diagnostic StudiesJournal of the American Medical Directors Association, 2018
- Spatial and dynamical handwriting analysis in mild cognitive impairmentComputers in Biology and Medicine, 2017
- Brain atrophy in Alzheimer’s Disease and agingAgeing Research Reviews, 2016
- The Diagnosis and Management of Mild Cognitive ImpairmentJAMA, 2014