Real-time Recognition of Contextual Composite Activities from Asynchronous Multi-IMU Sensor Streams
Open Access
- 5 November 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Sensor Networks
Abstract
Human Activity Recognition (HAR) is foundational to intelligent systems delivering personalized healthcare and wellbeing services. Wearable devices equipped with inertial measurement units (IMUs), such as smartwatches and earables, have enabled non-intrusive recognition of motion-driven activities. However, real-time recognition of contextual composite activities – temporally overlapping actions initiated by distinct body parts – remains challenging due to asynchronous multi-IMU data streams, where data collected from multiple IMU-enabled wearable devices may exhibit heterogeneous sampling rates, independent device clocks, and communication delays, resulting in temporally misaligned sensor readings, variability in sensor quality, and limited contextual cues from low-resolution streaming data. To address these challenges, we propose Multisource Self-Supervised Learning (MSSL), a lightweight framework that exploits unlabeled multisource-IMU data to enhance composite activity recognition in real time. MSSL incorporates (1) a dynamic source selection mechanism leveraging probability density estimation to adaptively identify primary and auxiliary sensor sources, and (2) a multimodal contrastive learning strategy to improve feature representations across heterogeneous sensor modalities. Evaluations on an in-house dataset and four benchmark HAR datasets demonstrate that MSSL achieves (i) 95.26% accuracy on our in-house composite activity dataset, (ii) 85.93% on PAMAP2-ADL, (iii) 80.08% on Opportunity, (iv) 91.18% on RealDisp, and (v) 69.41% on PAMAP2-Locomotions. Notably, MSSL maintains robust performance even with only 10% labeled data (78.05% accuracy) while achieving low inference latency of 0.6328 milliseconds. These results underscore MSSL’s potential for reliable, context-aware composite activity recognition in real-time applications, paving the way for more responsive and personalized services.Keywords
This publication has 38 references indexed in Scilit:
- Human activity recognition with smartphone-integrated sensors: A surveyExpert Systems with Applications, 2024
- A review of vision-based indoor HAR: state-of-the-art, challenges, and future prospectsMultimedia Tools and Applications, 2023
- Context-aware mutual learning for semi-supervised human activity recognition using wearable sensorsExpert Systems with Applications, 2023
- Efficient algorithms for victim item selection in privacy-preserving utility miningFuture Generation Computer Systems, 2022
- Towards fully verifiable forward secure privacy preserving keyword search for IoT outsourced dataFuture Generation Computer Systems, 2022
- Subject variability in sensor-based activity recognitionJournal of Ambient Intelligence and Humanized Computing, 2021
- Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniquesExpert Systems with Applications, 2020
- Adaptive sliding window based activity recognition for assisted livingsInformation Fusion, 2020
- A robust human activity recognition system using smartphone sensors and deep learningFuture Generation Computer Systems, 2018
- Human activity recognition with smartphone sensors using deep learning neural networksExpert Systems with Applications, 2016