A Turbo Q-Learning (TQL) for Energy Efficiency Optimization in Heterogeneous Networks
Open Access
Abstract
In order to maximize energy efficiency in heterogeneous networks (HetNets), a turbo Q-Learning (TQL) combined with multistage decision process and tabular Q-Learning is proposed to optimize the resource configuration. For the large dimensions of action space, the problem of energy efficiency optimization is designed as a multistage decision process in this paper, according to the resource allocation of optimization objectives, the initial problem is divided into several subproblems which are solved by tabular Q-Learning, and the traditional exponential increasing size of action space is decomposed into linear increase. By iterating the solutions of subproblems, the initial problem is solved. The simple stability analysis of the algorithm is given in this paper. As to the large dimension of state space, we use a deep neural network (DNN) to classify states where the optimization policy of novel Q-Learning is set to label samples. Thus far, the dimensions of action and state space have been solved. The simulation results show that our approach is convergent, improves the convergence speed by 60% while maintaining almost the same energy efficiency and having the characteristics of system adjustment.Keywords
This publication has 34 references indexed in Scilit:
- Internet of Things (IoT) for Next-Generation Smart Systems: A Review of Current Challenges, Future Trends and Prospects for Emerging 5G-IoT ScenariosIEEE Access, 2020
- Joint Subcarrier Assignment and Global Energy-Efficient Power Allocation for Energy-Harvesting Two-Tier Downlink NOMA HetnetsIEEE Access, 2019
- Channel Access and Power Control for Energy-Efficient Delay-Aware Heterogeneous Cellular Networks for Smart Grid Communications Using Deep Reinforcement LearningIEEE Access, 2019
- Compressive Sensing Based Spectrum Allocation and Power Control for NOMA HetNetsIEEE Access, 2019
- TOFFEE: Task Offloading and Frequency Scaling for Energy Efficiency of Mobile Devices in Mobile Edge ComputingIEEE Transactions on Cloud Computing, 2019
- DRAG: Deep Reinforcement Learning Based Base Station Activation in Heterogeneous NetworksIEEE Transactions on Mobile Computing, 2019
- Reinforcement Learning for Self Organization and Power Control of Two-Tier Heterogeneous NetworksIEEE Transactions on Wireless Communications, 2019
- Energy Saving and Interference Coordination in HetNets Using Dynamic Programming and CECIEEE Access, 2018
- Data-Driven Configuration of Interference Coordination Parameters in HetNetsIEEE Transactions on Vehicular Technology, 2018
- User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply: An Actor-Critic Reinforcement Learning ApproachIEEE Transactions on Wireless Communications, 2017