Abstract
The proliferation of low-cost, portable sensors has made wearable human activity recognition (HAR) a cornerstone for real-time health monitoring and behavior analysis. However, deploying accurate yet lightweight deep learning models on resource-constrained wearable devices poses a significant challenge for on-device activity recognition. While channel pruning is a common solution to accelerate deep convolutional neural networks (CNNs), existing works often require specialized implementations or pretrained models, which potentially degrade performance by simply removing an entire channel, limiting their ability to handle complex multimodal sensor inputs. Moreover, lightweight CNN design, particularly the heavy use of $1\times 1$ convolution layers for channel squeezing, remains inefficient for sensor-based HAR, which consumes resources without expanding the receptive field due to their pointwise nature. To address these issues, we propose a novel dynamic channel sampling module, select-and-fuse (SaF), specifically designed for sensor-based HAR. SaF divides channels into subsets and performs a dynamic, input-dependent selection from them, with the picking decision being made per time step based on the input sensor signal activations, allowing for fine-grained feature adaptation to multimodal sensor signals. While integrated into compact backbones, SaF significantly reduces model size and inference latency while maintaining high accuracy. Extensive evaluations on public UCI-HAR, OPPORTUNITY, WISDM, and UniMiB-SHAR benchmarks confirm a favorable performance–cost tradeoff. Crucially, we measure actual inference latency on a Raspberry Pi, proving its practicality for resource-constrained HAR applications. The code will be released.
Funding Information
  • National Natural Science Foundation of China (62373194)
  • Cultivating Plan Program for the Leader in Science and Technology of Yunnan Province, China (202005AC160005)
  • Ten Thousand Talent Plans for Young of Yunnan Province, China (YNWR-QNBJ-2019-188)