Automated Customization of On-Device Inference for Quality-of-Experience Enhancement
- 20 September 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 72 (5) , 1329-1342
Abstract
The rapid uptake of intelligent applications is pushing deep learning (DL) capabilities to mobile devices. However, the heterogeneities in device capacity, DNN performances, and user preferences make it challenging to provide satisfactory Quality of Experience (QoE) to mobile users. This paper studies automated customization for DL inference on mobile devices (termed as on-device inference), and our goal is to enhance user QoE by configuring the on-device inference with an appropriate DNN for users under different usage scenarios. The core of our method is a DNN selection module that learns user QoE patterns on-the-fly and identifies the best-fit DNN for on-device inference with the learned knowledge. It leverages an online learning algorithm, NeuralUCB , that has excellent generalization ability for handling various user QoE patterns. We also embed the knowledge transfer technique in NeuralUCB to expedite the learning process. However, NeuralUCB frequently solicits QoE ratings from users, which incurs non-negligible inconvenience. To address this problem, we design feedback solicitation schemes to reduce the number of QoE solicitations while maintaining the learning efficiency of NeuralUCB. A pragmatic problem, aggregated QoE , is further investigated to improve the practicality of our framework. We conduct experiments on both synthetic and real-world data. The results indicate that our method efficiently learns the user QoE pattern with few solicitations and provides drastic QoE enhancement for mobile devices.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62202293)
- National Science Foundation (CNS-2007115, 2006630, 2044991, 2029858, 2033681)
This publication has 29 references indexed in Scilit:
- Does network quality matter? A field study of mobile user satisfactionPervasive and Mobile Computing, 2017
- BranchyNet: Fast inference via early exiting from deep neural networks2016
- MCDNN2016
- Quantized Convolutional Neural Networks for Mobile Devices2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Exploring QoE in Cellular Networks2015
- From QoS to QoE: A Tutorial on Video Quality AssessmentIEEE Communications Surveys & Tutorials, 2014
- Prometheus2014
- Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets2013
- Incentive and informational properties of preference questionsEnvironmental and Resource Economics, 2007