Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges
Top Cited Papers
- 5 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Network
- Vol. 39 (1) , 189-196
Abstract
This article presents a digital twin (DT)-enhanced reinforcement learning (RL) framework aimed at optimizing performance and reliability in network resource management, since the traditional RL methods face several unified challenges when applied to physical networks, including limited exploration efficiency, slow convergence, poor long-term performance, and safety concerns during the exploration phase. To deal with the above challenges, a comprehensive DT-based framework is proposed to enhance the convergence speed and performance for unified RL-based resource management. The proposed framework provides safe action exploration, more accurate estimates of long-term returns, faster training convergence, higher convergence performance, and real-time adaptation to varying network conditions. Then, two case studies on ultrareliable and low-latency communication (URLLC) services and multiple unmanned aerial vehicles (UAV) network are presented, demonstrating improvements of the proposed framework in performance, convergence speed, and training cost reduction both on traditional RL and neural network based Deep RL (DRL). Finally, the article identifies and explores some of the research challenges and open issues in this rapidly evolving field.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2020YFB1807700)
This publication has 13 references indexed in Scilit:
- Dynamic Neural Network-Based Resource Management for Mobile Edge Computing in 6G NetworksIEEE Transactions on Cognitive Communications and Networking, 2023
- Transfer Learning in Deep Reinforcement Learning: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Imperfect Digital Twin Assisted Low Cost Reinforcement Training for Multi-UAV Networks2023
- A Dynamic Hierarchical Framework for IoT-Assisted Digital Twin Synchronization in the MetaverseIEEE Internet of Things Journal, 2022
- Exploration in deep reinforcement learning: A surveyInformation Fusion, 2022
- Holistic Network Virtualization and Pervasive Network Intelligence for 6GIEEE Communications Surveys & Tutorials, 2021
- Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A TutorialIEEE Communications Surveys & Tutorials, 2021
- Machine Learning for 6G Wireless Networks: Carrying Forward Enhanced Bandwidth, Massive Access, and Ultrareliable/Low-Latency ServiceIEEE Vehicular Technology Magazine, 2020
- Resource Management at the Network Edge: A Deep Reinforcement Learning ApproachIEEE Network, 2019
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998