Triple Spectral Fusion for Sensor-based Human Activity Recognition
- 6 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-18
Abstract
The field of sensor-based human activity recognition (HAR) mainly uses posture, motion and context data of Inertial Measurement Units (IMUs) to identify daily activities. Despite the advancements in learning-based methods, it is challenging to perform information fusion from the temporal perspective due to the complexities in fusing heterogeneous sensor data and establishing long-term context correlations. This paper proposes a novel triple spectral fusion framework tailored for HAR. First, we develop an adaptive complementary filtering technique for noise suppression and organize each IMU’s sensors into posture and motion modality nodes. Given that IMU nodes form a dynamic heterogeneous graph, we then apply adaptive filtering within the graph Fourier domain to merge both homogeneous and heterogeneous node information. Furthermore, an adaptive wavelet frequency selection approach is implemented to suppress context redundancy and shorten the length of features. This approach enhances both timestamp-based graph aggregation and the correlation of long-term contexts. Our framework uses adaptive filtering in the Fourier, graph Fourier, and wavelet domains, enabling effective multi-sensor fusion and context correlation. Extensive experiments on ten benchmark datasets demonstrate the superior performance of our framework. Project page: https://github.com/crocodilegogogo/TSF-TPAMI2026.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62372491)
- Aeronautical Science Fund of China (2025L0150M1001)
- Basic and Applied Basic Research Foundation of Guangdong Province (2023B1515120087)
- Science and Technology Planning Project of Key Laboratory of Advanced IntelliSense Technology
- Guangdong Science and Technology Department (2023B1212060024)
- Shenzhen Key Laboratory of Navigation and Communication Integration (ZDSYS20210623091807023)
- Shenzhen Science and Technology Program (ZDCY20250901100201002)
- Natural Science Foundation of Chongqing Municipality (CSTB2024NSCQ-JQX0020)
- Australian Research Council (DP260101891)
This publication has 59 references indexed in Scilit:
- Gesture recognition with adaptive-weight-based residual MultiheadCrossAttention fusion based on multi-level feature informationInformation Fusion, 2024
- IoT-FAR: A multi-sensor fusion approach for IoT-based firefighting activity recognitionInformation Fusion, 2024
- Multi-scale signed recurrence plot based time series classification using inception architectural networksPattern Recognition, 2022
- A review of multimodal human activity recognition with special emphasis on classification, applications, challenges and future directionsKnowledge-Based Systems, 2021
- Efficient dense labelling of human activity sequences from wearables using fully convolutional networksPattern Recognition, 2018
- Detecting unseen falls from wearable devices using channel-wise ensemble of autoencodersExpert Systems with Applications, 2017
- Bayesian Co-Boosting for Multi-modal Gesture Recognition2017
- Human Daily Activity and Fall Recognition Using a Smartphone’s Acceleration Sensor2017
- Transition-Aware Human Activity Recognition Using SmartphonesNeurocomputing, 2016
- Comparative study on classifying human activities with miniature inertial and magnetic sensorsPattern Recognition, 2010