Layout-Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST)
Open Access
- 3 March 2025
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 9 (1) , 1-38
Abstract
Human activity recognition (HAR) using ambient sensors in smart homes has numerous applications for human healthcare and wellness. However, building general-purpose HAR models that can be deployed to new smart home environments requires a significant amount of annotated sensor data and training overhead. Most smart homes vary significantly in their layouts, i.e., floor plans and the specifics of sensors embedded, resulting in low generalizability of HAR models trained for specific homes. We address this limitation by introducing a novel, layout-agnostic modeling approach for HAR systems in smart homes that utilizes the transferrable representational capacity of natural language descriptions of raw sensor data. To this end, we generate Textual Descriptions Of Sensor Triggers (TDOST) that encapsulate the surrounding trigger conditions and provide cues for underlying activities to the activity recognition models. Leveraging textual embeddings, rather than raw sensor data, we create activity recognition systems that predict standard activities across homes without (re-)training or adaptation to target homes. Through an extensive evaluation, we demonstrate the effectiveness of TDOST-based models in unseen smart homes through experiments on benchmark Orange4Home and CASAS datasets. Furthermore, we conduct a detailed analysis of how the individual components of our approach affect downstream activity recognition performance.Keywords
All Related Versions
Funding Information
- Cisco Systems (87483789)
- Optum (N/A)
- KDDI Research (N/A)
- National Science Foundation (2112633)
This publication has 77 references indexed in Scilit:
- A review of machine learning-based human activity recognition for diverse applicationsNeural Computing and Applications, 2022
- Embedding-based real-time change point detection with application to activity segmentation in smart home time series dataExpert Systems with Applications, 2021
- Sensor-based and vision-based human activity recognition: A comprehensive surveyPattern Recognition, 2020
- A sequential deep learning application for recognising human activities in smart homesNeurocomputing, 2020
- Unsupervised domain adaptation for activity recognition across heterogeneous datasetsPervasive and Mobile Computing, 2020
- Enhancing activity recognition using CPD-based activity segmentationPervasive and Mobile Computing, 2019
- Ensemble classifier of long short-term memory with fuzzy temporal windows on binary sensors for activity recognitionExpert Systems with Applications, 2018
- Dynamic detection of window starting positions and its implementation within an activity recognition frameworkJournal of Biomedical Informatics, 2016
- Dynamic sensor data segmentation for real-time knowledge-driven activity recognitionPervasive and Mobile Computing, 2014
- Genetic transfer learningExpert Systems with Applications, 2010