Conformalized Uncertainty Regions for Machine Learning-Based Multiple Cognitive Health Measures from Smartwatch Sensor Data
Open Access
- 16 May 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
- Vol. 7 (3) , 1-24
Abstract
Machine Learning (ML) algorithms play an increasingly critical role in remote health monitoring, automating health assessments, and extending medical professionals’ reach in caring for an aging population. However, ML models often provide predictions without quantifying uncertainty, which is essential for safe deployment in healthcare. This article presents a novel method referred to as Uncertainty Regions via Importance-Weighted Calibration (URIC) for automating the uncertainty-based prediction of multiple clinical cognitive health measures from continuous smartwatch sensor data. URIC leverages Conformal Prediction (CP), a rigorous Uncertainty Quantification (UQ) framework that guarantees user-defined coverage (e.g., ground truth is in the predicted region with 95% probability). The key innovation of URIC is constructing smaller, more interpretable prediction regions by assigning importance weights to calibration examples. It forms prediction regions by using the most relevant calibration examples, ensuring tight regions without sacrificing coverage. We evaluate URIC for predicting the cognitive health measures of 157 adults who were either cognitively healthy or experienced Mild Cognitive Impairment (MCI). Compared to other UQ methods, our method constructs smaller prediction regions across all multi-target combinations while maintaining the user-specified coverage. This approach can be integrated into human-ML collaborative systems to improve diagnostic accuracy and interpretability in sensitive multi-target healthcare tasks.Keywords
Funding Information
- National Institutes of Health (R01AG065218 and R35AG071451)
- National Science Foundation (1954372)
- Office of the Assistant Secretary for Health Affairs through the Peer-Reviewed Alzheimer’s Research Program (W81XWH-20-0654)
This publication has 29 references indexed in Scilit:
- Ensemble deep learning: A reviewEngineering Applications of Artificial Intelligence, 2022
- Localized conformal prediction: a generalized inference framework for conformal predictionBiometrika, 2022
- Conformal Prediction in Clinical Medical SciencesJournal of Healthcare Informatics Research, 2022
- Significance of machine learning in healthcare: Features, pillars and applicationsInternational Journal of Intelligent Networks, 2022
- Copula-based conformal prediction for multi-target regressionPattern Recognition, 2021
- Automated cognitive health assessment in smart homes using machine learningSustainable Cities and Society, 2021
- Development and Psychometric Properties of the Instrumental Activities of Daily Living: Compensation ScaleArchives of Clinical Neuropsychology, 2014
- Self-reported cognitive concerns and abilities: two sides of one coin?Psycho-Oncology, 2014
- The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008Journal of Clinical Epidemiology, 2010
- The PHQ-9Journal of General Internal Medicine, 2001