Scheduling Real-Time Wireless Traffic: A Network-Aided Offline Reinforcement Learning Approach
- 14 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 10 (24) , 22331-22340
Abstract
Real-time traffic has stringent requirements in terms of latency, and deadline guarantees on packet delivery play a vital role in real-time IoT applications. Deadline-aware wireless scheduling of real-time traffic has been a long-standing open problem, despite significant efforts using analytical methods. Departing from the conventional approaches, this work studies deadline-aware traffic scheduling by taking an offline reinforcement learning (RL) approach to train scheduling algorithms, ready to be used for online scheduling. To address the challenges therein, we propose a Network-Aided Offline RL (NA-ORL) framework for deadline-aware scheduling, by making use of the fact that the network dynamics follows a well-defined physics model. Specifically, in NA-ORL the initialization of the scheduling policy is obtained through behavior cloning with a good model-based scheduling algorithm, and the network-aided actor-critic (A-C) method is utilized to train a better scheduling policy with carefully designed states and reward function, thanks to its nature of policy improvement. Building on NA-ORL, we further devise a Network-Aided Offline Meta-RL (NA-MRL) algorithm to deal with the non-stationary network dynamics. Extensive experimental results demonstrate that the proposed NA-ORL and NA-MRL algorithms can achieve better performance over Adaptive Mixing over Non-Dominated links (AMIX-ND) and Largest-Deficit-First (LDF), in various scenarios for the deadline-aware wireless scheduling.Keywords
Funding Information
- NSF (CNS-2203239, CIF-2148253, CNS-2203412)
This publication has 17 references indexed in Scilit:
- Mastering the game of Go without human knowledgeNature, 2017
- Multiobjective Optimization for Stiffness and Position Control in a Soft Robot Arm ModuleIEEE Robotics and Automation Letters, 2017
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- On the Capacity Requirement of Largest-Deficit-First for Scheduling Real-Time Traffic in Wireless Networks2015
- On the Performance of Largest-Deficit-First for Scheduling Real-Time Traffic in Wireless NetworksIEEE/ACM Transactions on Networking, 2014
- Scheduling Heterogeneous Real-Time Traffic Over Fading Wireless ChannelsIEEE/ACM Transactions on Networking, 2013
- Scheduling for Optimal Rate Allocation in Ad Hoc Networks With Heterogeneous Delay ConstraintsIEEE Journal on Selected Areas in Communications, 2011
- Utility-optimal scheduling in time-varying wireless networks with delay constraints2010
- Optimal Scheduling for Fair Resource Allocation in Ad Hoc Networks with Elastic and Inelastic Traffic2010
- A Theory of QoS for Wireless2009