Learning Sensor Sample-Reweighting for Dynamic Early-Exit Activity Recognition Via Meta Learning
- 31 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (6) , 4081-4094
Abstract
During recent years, dynamic early-exit has provided a promising paradigm to improve the computational efficiency of deep neural networks by constructing multiple classifiers to let easy samples exit at shallow layers while avoiding redundant computations at deep exits, which has been seldom explored in the context of latency-aware human activity recognition (HAR) deployed on wearable devices. Particularly, most existing early-exit strategies have always treated all activity samples equally at each exit during training, which ignore such dynamic early-exit behavior at test-time, causing a potential mismatch between training and test. Intuitively, easy activity samples that often exit earlier at test-time should place more emphasis on the training loss of shallow classifiers, while hard activity samples should contribute more to the training loss of deep classifiers. To bridge this gap, this paper introduces a sample-reweighting approach for efficient activity inference, which employs a weight-predicting network to reweight the training loss of different activity samples at every exit. From a perspective of meta learning, a new optimization objective function is designed to jointly optimize both weight-predicting network and backbone network. We perform extensive experiments on three popular HAR benchmarks including UCI-HAR, WISDM, and UniMiB-SHAR, which demonstrate that while incorporating such test-time early-exit behavior into conventional training pipeline, it can consistently improve the accuracy-efficiency trade-offs under budgeted batch classification and anytime prediction patterns. Moreover, our approach has a natural advantage in handing class-imbalance HAR problem. Detailed ablation studies, visualized illustrations, and real hardware deployment are provided to support our statement. Code will be available at https://github.com/FuZenan/LtSRW-for-EEAR .Keywords
Funding Information
- National Natural Science Foundation of China (62373194)
- Cultivating Plan Program for the Leader in Science and Technology of Yunnan Province, China (202005AC160005)
- Ten Thousand Talent Plans for Young of Yunnan Province, China (YNWR-QNBJ-2019-188)
This publication has 33 references indexed in Scilit:
- FreqSense: Adaptive Sampling Rates for Sensor-Based Human Activity Recognition Under Tunable Computational BudgetsIEEE Journal of Biomedical and Health Informatics, 2023
- Do we need early exit networks in human activity recognition?Engineering Applications of Artificial Intelligence, 2023
- A Novel PPG-FMG-ACC Wristband for Hand Gesture RecognitionIEEE Journal of Biomedical and Health Informatics, 2022
- SkipNet: Learning Dynamic Routing in Convolutional Networks2018
- Distilling the Knowledge From Handcrafted Features for Human Activity RecognitionIEEE Transactions on Industrial Informatics, 2018
- Densely Connected Convolutional Networks2017
- Human activity recognition with smartphone sensors using deep learning neural networksExpert Systems with Applications, 2016
- Deep Residual Learning for Image Recognition2016
- Introducing a New Benchmarked Dataset for Activity Monitoring2012
- Comparative study on classifying human activities with miniature inertial and magnetic sensorsPattern Recognition, 2010