Slicing Resource Allocation Based on Dueling DQN for eMBB and URLLC Hybrid Services in Heterogeneous Integrated Networks
Open Access
Abstract
In 5G/B5G communication systems, network slicing is utilized to tackle the problem of the allocation of network resources for diverse services with changing demands. We proposed an algorithm that prioritizes the characteristic requirements of two different services and tackles the problem of allocation and scheduling of resources in the hybrid services system with eMBB and URLLC. Firstly, the resource allocation and scheduling are modeled, subject to the rate and delay constraints of both services. Secondly, the purpose of adopting a dueling deep Q network (Dueling DQN) is to approach the formulated non-convex optimization problem innovatively, in which a resource scheduling mechanism and the -greedy strategy were utilized to select the optimal resource allocation action. Moreover, the reward-clipping mechanism is introduced to enhance the training stability of Dueling DQN. Meanwhile, we choose a suitable bandwidth allocation resolution to increase flexibility in resource allocation. Finally, the simulations indicate that the proposed Dueling DQN algorithm has excellent performance in terms of quality of experience (QoE), spectrum efficiency (SE) and network utility, and the scheduling mechanism makes the performance much more stable. In contrast with Q-learning, DQN as well as Double DQN, the proposed algorithm based on Dueling DQN improves the network utility by 11%, 8% and 2%, respectively.
Keywords
Funding Information
- National Natural Science Foundation of China (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Natural Science Foundation of Shandong Province of China (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Talented Young Teachers Training Program of Shandong University of Science and Technology (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Innovative Research Foundation of Qingdao (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Elite Plan Project of Shandong University of Science and Technology (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- National Key R&D Program of China (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Hundred Talent Program of Chinese Academy of Sciences (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Sci. & Tech. Development Fund of Shandong Province of China (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Humanities and Social Science Research Project of the Ministry of Education (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Taishan Scholar Program of Shandong Province (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Shandong Chongqing Science and technology cooperation project (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Sci. & Tech. Development Fund of Qingdao (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- SDUST Research Fund (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
- Science and Technology Support Plan of Youth Innovation Team of Shandong Higher School (61701284, 61871370, ZR2022MF226, BJ20221101, 19-6-2-1-cg, skr21-3-B-048, 2019YFE0120700, 2019YFB1803101, E06BRA1001, ZR202102230289, ZR202102250695, ZR2019LZH001, 18YJAZH017, ts20190936, cstc2020jscx-lyjsAX0008, 21-1-5-zlyj-1-zc, 2015TDJH102, 2019KJN024)
This publication has 37 references indexed in Scilit:
- eMBB-URLLC Resource Slicing: A Risk-Sensitive ApproachIEEE Communications Letters, 2019
- 5G RAN Slicing for Verticals: Enablers and ChallengesIEEE Communications Magazine, 2019
- Deep Learning in Mobile and Wireless Networking: A SurveyIEEE Communications Surveys & Tutorials, 2019
- Deep Reinforcement Learning for Resource Management in Network SlicingIEEE Access, 2018
- Deep Learning for Intelligent Wireless Networks: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2018
- Multi-User Preemptive Scheduling For Critical Low Latency Communications in 5G Networks2018
- Bandwidth Efficiency and Service Adaptiveness Oriented Data Dissemination in Heterogeneous Vehicular NetworksIEEE Transactions on Vehicular Technology, 2018
- Network Slicing in 5G: Survey and ChallengesIEEE Communications Magazine, 2017
- Resource Slicing in Virtual Wireless Networks: A SurveyIEEE Transactions on Network and Service Management, 2016
- Quality-oriented Rate Control and Resource Allocation in Time-Varying OFDMA NetworksIEEE Transactions on Vehicular Technology, 2016