A ${Q}$ -Learning Based Framework for Congested Link Identification
- 23 July 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 6 (6) , 9668-9678
Abstract
Network congestion will result in significant performance degradation or even failures of many bandwidth-hungry IoT applications. Accurate and efficient congested link identification has become a foundational issue to IoT applications like self-driving cars, digital health, smart city, and so on. However, directly monitoring the massive number of interior links often introduces high operation cost or even is infeasible in practice, giving rise to indirect monitoring techniques like network boolean tomography. Nevertheless, in many networks, the number of their interior links is larger than their end-to-end paths, making it very challenging for network boolean tomography to find a determined solution. To resolve this issue, most of current methods try to utilize some prerequisites, such as the link congestion probabilities. While these probabilities might be hard or even unable to be obtained accurately in dynamical networks, limiting the practical deployment. In this paper, we are motivated to design a framework of congested link identification without any prerequisite or assumption. We first novelly model the congested link identification procedures as a Markov decision processes (MDP), and then employ a reinforcement learning technology, i.e., Q-learning, to solve this MDP. The simulation results show that our proposed scheme can autonomously and efficiently explore the unknown network environment, and is able to achieve better adaptivity and correctness, without any prior knowledge comparing to existing methods.Keywords
Funding Information
- National Basic Research Program of China (2018YFB1003500)
- National Natural Science Foundation of China (61701074, 61772480, 61872310)
- Japan Society for the Promotion of Science (19K20258)
- China University of Geosciences, Wuhan (G1323541861)
This publication has 39 references indexed in Scilit:
- Human-level control through deep reinforcement learningNature, 2015
- Notice of Removal: Peer-to-Peer Compressive Sensing for Network MonitoringIEEE Communications Letters, 2014
- A Vision of IoT: Applications, Challenges, and Opportunities With China PerspectiveIEEE Internet of Things Journal, 2014
- Monitor placement for maximal identifiability in network tomography2014
- An Information Framework for Creating a Smart City Through Internet of ThingsIEEE Internet of Things Journal, 2014
- The design space of probing algorithms for network-performance measurementACM SIGMETRICS Performance Evaluation Review, 2013
- Toward accurate and practical network tomographyACM SIGOPS Operating Systems Review, 2013
- Locating congested segments over the Internet by clustering the delay performance of multiple pathsComputer Communications, 2009
- A two-layered multi-agent reinforcement learning model and algorithmJournal of Network and Computer Applications, 2007
- GRE encapsulated multicast probingACM SIGMETRICS Performance Evaluation Review, 2007