Adaptive UAV-Trajectory Optimization Under Quality of Service Constraints: A Model-Free Solution
Open Access
- 11 June 2020
- journal article
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 112253-112265
Abstract
Unmanned aerial vehicles (UAVs) with the potential of providing reliable high-rate connectivity, are becoming a promising component of future wireless networks. A UAV collects data from a set of randomly distributed sensors, where both the locations of these sensors and their data volume to be transmitted are unknown to the UAV. In order to assist the UAV in finding the optimal motion trajectory in the face of the uncertainty without the above knowledge whilst aiming for maximizing the cumulative collected data, we formulate a reinforcement learning problem by modelling the motion-trajectory as a Markov decision process with the UAV acting as the learning agent. Then, we propose a pair of novel trajectory optimization algorithms based on stochastic modelling and reinforcement learning, which allows the UAV to optimize its flight trajectory without the need for system identification. More specifically, by dividing the considered region into small tiles, we conceive state-action-reward-state-action (Sarsa) and Q-learning based UAV-trajectory optimization algorithms (i.e., SUTOA and QUTOA) aiming to maximize the cumulative data collected during the finite flight-time. Our simulation results demonstrate that both of the proposed approaches are capable of finding an optimal trajectory under the flight-time constraint. The preference for QUTOA vs. SUTOA depends on the relative position of the start and the end points of the UAVs.Keywords
Funding Information
- Engineering and Physical Sciences Research Council (EP/Noo4558/1, EP/PO34284/1)
- COALESCE
- Royal Society’s Global Challenges Research Fund
- European Research Council’s Advanced Fellowship under Grant QuantCom
- U.K. Engineering and Physical Sciences Research Council (EP/N029720/2)
This publication has 34 references indexed in Scilit:
- Caching in the Sky: Proactive Deployment of Cache-Enabled Unmanned Aerial Vehicles for Optimized Quality-of-ExperienceIEEE Journal on Selected Areas in Communications, 2017
- Charging Unplugged: Will Distributed Laser Charging for Mobile Wireless Power Transfer Work?IEEE Vehicular Technology Magazine, 2016
- Throughput Maximization for UAV-Enabled Mobile Relaying SystemsIEEE Transactions on Communications, 2016
- Designing and implementing future aerial communication networksIEEE Communications Magazine, 2016
- Optimal LAP Altitude for Maximum CoverageIEEE Wireless Communications Letters, 2014
- Model-Free Trajectory Optimisation for Unmanned Aircraft Serving as Data Ferries for Widespread SensorsRemote Sensing, 2012
- Speed control and scheduling of data mules in sensor networksACM Transactions on Sensor Networks, 2010
- Reinforcement learning with replacing eligibility tracesMachine Learning, 1996
- On the Convergence of Stochastic Iterative Dynamic Programming AlgorithmsNeural Computation, 1994
- TD(?) converges with probability 1Machine Learning, 1994