FeDRL-D2D: Federated Deep Reinforcement Learning- Empowered Resource Allocation Scheme for Energy Efficiency Maximization in D2D-Assisted 6G Networks
Open Access
- 29 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 109775-109792
Abstract
Device-to-device (D2D)-assisted 6G networks are expected to support the proliferation of ubiquitous mobile applications by enhancing system capacity and overall energy efficiency towards a connected-sustainable world. However, the stringent quality of service (QoS) requirements for ultra-massive connectivity, high mobility, limited network resources, and interference management are the significant challenges to deploying multiple device-to-device pairs (DDPs) without disrupting cellular users. Hence, intelligent resource management and power control are indispensable for alleviating interference among DDPs to optimize overall system performance and global energy efficiency. Considering this, we present a Federated DRL-based method for energy-efficient resource management in a D2D-assisted heterogeneous network (HetNet). We formulate a joint optimization problem of power control and channel allocation to maximize the system’s energy efficiency under QoS constraints for cellular user equipment (CUEs) and DDPs. The proposed scheme employs federated learning for a decentralized training paradigm to address user privacy, and a double-deep Q-network (DDQN) is used for intelligent resource management. The proposed DDQN method uses two separate Q-networks for action selection and target estimation to rationalize the transmit power and dynamic channel selection in which DDPs as agents could reuse the uplink channels of CUEs. Simulation results depict that the proposed method improves the overall system energy efficiency by 41.52% and achieves a better sum rate of 11.65%, 24.78%, and 47.29% than multi-agent actor-critic (MAAC), distributed deep-deterministic policy gradient (D3PG), and deep Q network (DQN) scheduling, respectively. Moreover, the proposed scheme achieves a 5.88%, 15.79%, and 27.27% reduction in cellular outage probability compared to MAAC, D3PG, and DQN scheduling, respectively, highlighting the scalability and robustness of the proposed solution.Keywords
Funding Information
- Ministry of Higher Education Malaysia under the Fundamental Research Grant Scheme (FRGS/1/2020/TK0/UM/02/30)
This publication has 54 references indexed in Scilit:
- Federated learning for green and sustainable 6G IIoT applicationsInternet of Things, 2024
- Federated learning for resource allocation in vehicular edge computing-enabled moving small cell networksVehicular Communications, 2023
- FedSSC: Joint client selection and resource management for communication-efficient federated vehicular networksComputer Networks, 2023
- Deep reinforcement learning based trajectory design and resource allocation for task-aware multi-UAV enabled MEC networksComputer Communications, 2023
- Deep reinforcement learning empowered joint mode selection and resource allocation for RIS-aided D2D communicationsNeural Computing and Applications, 2023
- Artificial Intelligence and Machine Learning in the Era of 5G and 6G Technology2023
- A survey of applications of artificial intelligence and machine learning in future mobile networks-enabled systemsEngineering Science and Technology, an International Journal, 2023
- Multi-agent reinforcement learning based joint uplink–downlink subcarrier assignment and power allocation for D2D underlay networksWireless Networks, 2022
- 6G Mobile Communication Technology: Requirements, Targets, Applications, Challenges, Advantages, and OpportunitiesAlexandria Engineering Journal, 2022
- Introduction to Reinforcement and Systemic Machine Learning2012