Joint Speed Control and Energy Replenishment Optimization for UAV-Assisted IoT Data Collection With Deep Reinforcement Transfer Learning
- 14 February 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 10 (7) , 5778-5793
Abstract
Unmanned aerial vehicle (UAV)-assisted data collection has been emerging as a prominent application due to its flexibility, mobility, and low operational cost. However, under the dynamic and uncertainty of IoT data collection and energy replenishment processes, optimizing the performance for UAV collectors is a very challenging task. Thus, this paper introduces a novel framework that jointly optimizes the flying speed and energy replenishment for each UAV to significantly improve the overall system performance (e.g., data collection and energy usage efficiency). Specifically, we first develop a Markov decision process to help the UAV automatically and dynamically make optimal decisions under the dynamics and uncertainties of the environment. Although traditional reinforcement learning algorithms such as Q-learning and deep Q-learning can help the UAV to obtain the optimal policy, they often take a long time to converge and require high computational complexity. Therefore, it is impractical to deploy these conventional methods on UAVs with limited computing capacity and energy resource. To that end, we develop advanced transfer learning techniques that allow UAVs to “share” and “transfer” learning knowledge, thereby reducing the learning time as well as significantly improving learning quality. Extensive simulations demonstrate that our proposed solution can improve the average data collection performance of the system up to 200% and reduce the convergence time up to 50% compared with those of conventional methods.Keywords
All Related Versions
Funding Information
- Australian Government through the Australian Research Council’s Discovery Projects funding scheme (DE210100651)
- Vingroup Science and Technology Scholarship Program for Overseas Study for Master’s and Doctoral Degrees
- University of Transport and Communications, Hanoi, Vietnam
This publication has 25 references indexed in Scilit:
- Joint UAV Hovering Altitude and Power Control for Space-Air-Ground IoT NetworksIEEE Internet of Things Journal, 2018
- Common Throughput Maximization in UAV-Enabled OFDMA Systems With Delay ConsiderationIEEE Transactions on Communications, 2018
- Energy-Efficient UAV Control for Effective and Fair Communication Coverage: A Deep Reinforcement Learning ApproachIEEE Journal on Selected Areas in Communications, 2018
- Flight Time Minimization of UAV for Data Collection Over Wireless Sensor NetworksIEEE Journal on Selected Areas in Communications, 2018
- Energy Efficient IoT Sensor With RF Wake-Up and Addressing CapabilityIEEE Sensors Letters, 2017
- 'Connecting the world from the sky'2016
- Human-level control through deep reinforcement learningNature, 2015
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- Q-learningMachine Learning, 1992
- A Stochastic Approximation MethodThe Annals of Mathematical Statistics, 1951