Packet Flow Capacity Autonomous Operation Based on Reinforcement Learning
Open Access
Abstract
As the dynamicity of the traffic increases, the need for self-network operation becomes more evident. One of the solutions that might bring cost savings to network operators is the dynamic capacity management of large packet flows, especially in the context of packet over optical networks. Machine Learning, particularly Reinforcement Learning, seems to be an enabler for autonomicity as a result of its inherent capacity to learn from experience. However, precisely because of that, RL methods might not be able to provide the required performance (e.g., delay, packet loss, and capacity overprovisioning) when managing the capacity of packet flows, until they learn the optimal policy. In view of that, we propose a management lifecycle with three phases: (i) a self-tuned threshold-based approach operating just after the packet flow is set up and until enough data on the traffic characteristics are available; (ii) an RL operation based on models pre-trained with a generic traffic profile; and (iii) an RL operation with models trained for real traffic. Exhaustive simulation results confirm the poor performance of RL algorithms until the optimal policy is learnt and when traffic characteristics change over time, which prevents deploying such methods in operators’ networks. In contrast, the proposed lifecycle outperforms benchmarking approaches, achieving noticeable performance from the beginning of operation while showing robustness against traffic changes.Keywords
Funding Information
- European Commission (101016663)
- Australian Education International, Australian Government (PID2020-114135RB-I00)
- Institució Catalana de Recerca i Estudis Avançats (-)
This publication has 23 references indexed in Scilit:
- Building Autonomic Elastic Optical Networks with Deep Reinforcement LearningIEEE Communications Magazine, 2019
- Monitoring and Data Analytics for Optical Networking: Benefits, Architectures, and Use CasesIEEE Network, 2019
- DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical NetworksJournal of Lightwave Technology, 2019
- Learning Life Cycle to Speed Up Autonomic Optical Transmission and Networking AdoptionJournal of Optical Communications and Networking, 2019
- A Data-Driven Bandwidth Allocation Framework With QoS Considerations for EONsJournal of Lightwave Technology, 2019
- Machine Learning for Network Automation: Overview, Architecture, and Applications [Invited Tutorial]Journal of Optical Communications and Networking, 2018
- CURSA-SQ: A Methodology for Service-Centric Traffic Flow AnalysisJournal of Optical Communications and Networking, 2018
- Future mode of operations for 5G – The SELFNET approach enabled by SDN/NFVComputer Standards & Interfaces, 2017
- An architecture to support autonomic slice networkingJournal of Lightwave Technology, 2017
- Introduction to Time Series and Forecasting2016