MarNASNets: Toward CNN Model Architectures Specific to Sensor-Based Human Activity Recognition
- 11 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 23 (16) , 18708-18717
Abstract
Deep learning (DL) models for sensor-based human activity recognition (HAR) are still in their nascent stages compared with image recognition. HAR’s inference is generally implemented on edge devices such as smartphones because of the secure privacy. However, lightweight DL models for HAR while meeting the hardware limitations are lacking. In this study, using the neural architecture search (NAS), we investigated an effective DL model architectures that can be used for inference on smartphones. We designed multiple search spaces for the type of convolution, the kernel size of the convolution process, the type of skip operation, the number of layers, and the number of output filters by Bayesian optimization. We propose models called mobile-aware convolutional neural network (CNN) for sensor-based HAR by NAS (MarNASNets). We constructed four MarNASNet networks, MarNASNets-A to D, each with a different model size and a parameter search space of four patterns. Experimental results show that MarNASNets achieve the same accuracy as existing CNN architectures with fewer parameters and are effective model architectures for on-device and sensor-based HAR. We also developed Activitybench, an iOS app, for measuring model performance on smartphones, and evaluated the on-device performance of each model. MarNASNets explored achieved accuracy comparable to existing CNN models with smaller model sizes. MarNASNet-C achieved accuracies of 92.60, 94.52, and 88.92 % for HASC, UCI, and WISDM, respectively. Especially for HASC and UCI, MarNASNet-C achieved the highest accuracies despite the small model size. Their latency was also comparable to that of existing CNN models, enabling real-time on-device inference.Keywords
Funding Information
- the Japan Society for the Promotion of Science (JSPS) KAKENHI Grant-in-Aid for Early-Career Scientists and Grant-in-Aid for Scientific Research (19K20420, 23K11164)
This publication has 36 references indexed in Scilit:
- Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity RecognitionSensors, 2016
- A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer2015
- Going deeper with convolutions2015
- Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors2014
- Shaka: User Movement Estimation Considering Reliability, Power Saving, and Latency Using Mobile PhoneIEICE Transactions on Information and Systems, 2011
- Objectively Measured Moderate- and Vigorous-Intensity Physical Activity but Not Sedentary Time Predicts Insulin Resistance in High-Risk IndividualsDiabetes Care, 2009
- Wearable Sensors for Reliable Fall Detection2005
- Digitizing Personal Experiences: Capture and Retrieval of Life Log2005
- Activity Recognition from User-Annotated Acceleration Data2004
- Physical activity patterns in normal, overweight and obese individuals using minute-by-minute accelerometryEuropean Journal of Clinical Nutrition, 2000