Abstract
Machine Learning (ML) algorithms play an increasingly critical role in remote health monitoring, automating health assessments, and extending medical professionals’ reach in caring for an aging population. However, ML models often provide predictions without quantifying uncertainty, which is essential for safe deployment in healthcare. This article presents a novel method referred to as Uncertainty Regions via Importance-Weighted Calibration (URIC) for automating the uncertainty-based prediction of multiple clinical cognitive health measures from continuous smartwatch sensor data. URIC leverages Conformal Prediction (CP), a rigorous Uncertainty Quantification (UQ) framework that guarantees user-defined coverage (e.g., ground truth is in the predicted region with 95% probability). The key innovation of URIC is constructing smaller, more interpretable prediction regions by assigning importance weights to calibration examples. It forms prediction regions by using the most relevant calibration examples, ensuring tight regions without sacrificing coverage. We evaluate URIC for predicting the cognitive health measures of 157 adults who were either cognitively healthy or experienced Mild Cognitive Impairment (MCI). Compared to other UQ methods, our method constructs smaller prediction regions across all multi-target combinations while maintaining the user-specified coverage. This approach can be integrated into human-ML collaborative systems to improve diagnostic accuracy and interpretability in sensitive multi-target healthcare tasks.
Funding Information
  • National Institutes of Health (R01AG065218 and R35AG071451)
  • National Science Foundation (1954372)
  • Office of the Assistant Secretary for Health Affairs through the Peer-Reviewed Alzheimer’s Research Program (W81XWH-20-0654)

This publication has 29 references indexed in Scilit: