Analyzing Sensor-Based Individual and Population Behavior Patterns via Inverse Reinforcement Learning
Open Access
Abstract
Digital markers of behavior can be continuously created, in everyday settings, using time series data collected by ambient sensors. The goal of this work was to perform individual- and population-level behavior analysis from such time series sensor data. In this paper, we introduce a novel algorithm—Resident Relative Entropy-Inverse Reinforcement Learning (RRE-IRL)—to perform an analysis of a single smart home resident or a group of residents, using inverse reinforcement learning. By employing this method, we learnt an individual’s behavioral routine preferences. We then analyzed daily routines for an individual and for eight smart home residents grouped by health diagnoses. We observed that the behavioral routine preferences changed over time. Specifically, the probability that the observed behavior was the same at the beginning of data collection as it was at the end (months later) was lower for residents experiencing cognitive decline than for cognitively healthy residents. When comparing aggregated behavior between groups of residents from the two diagnosis groups, the behavioral difference was even greater. Furthermore, the behavior preferences were used by a random forest classifier to predict a resident’s cognitive health diagnosis, with an accuracy of 0.84.Keywords
Funding Information
- National Institutes of Health (R01EB009675)
This publication has 40 references indexed in Scilit:
- Health behavior models for informing digital technology interventions for individuals with mental illness.Psychiatric Rehabilitation Journal, 2017
- Imitation LearningACM Computing Surveys, 2017
- Variability in medication taking is associated with cognitive performance in nondemented older adultsAlzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 2017
- Dynamic socialized Gaussian process models for human behavior prediction in a health social networkKnowledge and Information Systems, 2015
- Unobtrusive Detection of Mild Cognitive Impairment in Older Adults Through Home MonitoringIEEE Journal of Biomedical and Health Informatics, 2015
- Early Detection and Treatment of Type 2 Diabetes Reduce Cardiovascular Morbidity and Mortality: A Simulation of the Results of the Anglo-Danish-Dutch Study of Intensive Treatment in People With Screen-Detected Diabetes in Primary Care (ADDITION-Europe)Diabetes Care, 2015
- Evidence for a pervasive ‘idling-mode’ activity template in flying and pedestrian insectsRoyal Society Open Science, 2015
- Predicting human mobility with activity changesInternational Journal of Geographical Information Science, 2015
- Uncovering individual and collective human dynamics from mobile phone recordsJournal of Physics A: Mathematical and Theoretical, 2008
- On Information and SufficiencyThe Annals of Mathematical Statistics, 1951