Learning-Based and Data-Driven TCP Design for Memory-Constrained IoT
- 1 May 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 199-205
Abstract
Advances in wireless technology have resulted in pervasive deployment of devices of a high variability in form factors, memory and computational ability. The need for maintaining continuous connections that deliver data with high reliability necessitate re-thinking of conventional design of the transport layer protocol. This paper investigates the use of Q-learning in TCP cwnd adaptation during the congestion avoidance state, wherein the classical alternation of the window is replaced, thereby allowing the protocol to immediately respond to previously seen network conditions. Furthermore, it demonstrates how memory plays a critical role in building the exploration space, and proposes ways to reduce this overhead through function approximation. The superior performance of the learning-based approach over TCP New Reno is demonstrated through a comprehensive simulation study, revealing 33.8% and 12.1% improvement in throughput and delay, respectively, for the evaluated topologies. We also show how function approximation can be used to dramatically reduce the memory requirements of a learning-based protocol while maintaining the same throughput and delay.Keywords
This publication has 14 references indexed in Scilit:
- An experimental study of the learnability of congestion controlACM SIGCOMM Computer Communication Review, 2014
- RFID Technology for IoT-Based Personal Healthcare in Smart SpacesIEEE Internet of Things Journal, 2014
- TCP ex machinaACM SIGCOMM Computer Communication Review, 2013
- A Machine Learning Approach to End-to-End RTT Estimation and its Application to TCP2011
- Multi-resolution state-space discretization for Q-Learning with pseudo-randomized discretization2010
- A Machine Learning Approach to TCP Throughput PredictionIEEE/ACM Transactions on Networking, 2010
- Loss classification in optical burst switching networks using machine learning techniques: improving the performance of TCPIEEE Journal on Selected Areas in Communications, 2008
- CitySense: An Urban-Scale Wireless Sensor Network and Testbed2008
- Sparse Distributed Memories for On-Line Value-Based Reinforcement Learning2004
- Residual Algorithms: Reinforcement Learning with Function Approximation1995