Deep Reinforcement Learning for Smart City Communication Networks
- 1 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 17 (6) , 4188-4196
Abstract
Smart city communication networks hold the promise of harnessing the Internet of Things, smart devices, intelligent energy grids, autonomous cars and many other data architectures to improve quality of life for every citizen. And, as time and progress have gradually shifted this concept from an idea into a reality, the possibilities for what services a smart city might provide have exploded. Along with that expansion, the number of devices that may need to be supported across a communications network has multiplied enormously. However, existing congestion control protocols are not equipped to support this higher requirement for network performance. New protocols are needed to provide higher data throughputs, reduce queuing delays and packet loss, and maintain stable, reliable communications pathways. The end-to-end congestion control protocol presented in this paper does all these things. Called high throughput congestion control (HTCC), the framework takes full advantage of reinforcement learning and a fast growth algorithm to adaptively control the bytes in flight across the network's links to suit the prevailing network conditions. The result is a protocol that not only finds the optimal tradeoff between throughput, latency and packet loss, but also significantly speeds up the learning process to avoid wasting network resources, use all free bandwidth and maximize network utilization. A comprehensive set of experiments with HTCC and six state-of-the-art protocols -- Copa, PCC Vivace, PCC-Allegro, TCP Cubic, TaoVA-100x and FillP-Sheep -- demonstrate that, in most conditions, HTCC is able to make the best tradeoff between throughput, delay and loss rate.Keywords
Funding Information
- National Natural Science Foundation of China (61772377, 61672257, 91746206, 61972296, 61702380)
- Fundamental Research Funds for the Central Universities (2042020kf0217)
- Science and Technology Planning Project of Shenzhen (JCYJ20170818112550194)
- Wuhan Advanced Application (2019010701011419)
This publication has 14 references indexed in Scilit:
- A Survey of Delay-Based and Hybrid TCP Congestion Control AlgorithmsIEEE Communications Surveys & Tutorials, 2019
- Copa2018
- Mastering the game of Go without human knowledgeNature, 2017
- Applicability of Big Data Techniques to Smart Cities DeploymentsIEEE Transactions on Industrial Informatics, 2016
- An experimental study of the learnability of congestion control2014
- TCP ex machina2013
- CUBICACM SIGOPS Operating Systems Review, 2008
- FAST TCP: fairness and queuing issuesIEEE Communications Letters, 2005
- Improving the start-up behavior of a congestion control scheme for TCPACM SIGCOMM Computer Communication Review, 1996
- TCP Vegas: end to end congestion avoidance on a global InternetIEEE Journal on Selected Areas in Communications, 1995