Multi-Satellite Beam Hopping and Power Allocation Using Deep Reinforcement Learning
Authors:
- Xia XieXXXia Xie
- Kexin FanKFKexin Fan
- Wenfeng DengWDWenfeng Deng
- Nikolaos PappasNikolaos Pappas
- Qinyu ZhangQinyu Zhang
Preprint
Abstract
In non-geostationary orbit (NGSO) satellite communication systems, effectively utilizing beam hopping (BH) technology is crucial for addressing uneven traffic demands. However, optimizing beam scheduling and resource allocation in multi-NGSO BH scenarios remains a significant challenge. This paper proposes a multi-NGSO BH algorithm based on deep reinforcement learning (DRL) to optimize beam illumination patterns and power allocation. By leveraging three degrees of freedom (i.e., time, space, and power), the algorithm aims to optimize the long-term throughput and the long-term cumulative average delay (LTCAD). The solution is based on proximal policy optimization (PPO) with a hybrid action space combining discrete and continuous actions. Using two policy networks with a shared base layer, the proposed algorithm jointly optimizes beam scheduling and power allocation. One network selects beam illumination patterns in the discrete action space, while the other manages power allocation in the continuous space. Simulation results show that the proposed algorithm significantly reduces LTCAD while maintaining high throughput in time-varying traffic scenarios. Compared to the four benchmark methods, it improves network throughput by up to $8.9\%$ and reduces LTCAD by up to $69.2\%$
Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Multiobjective Optimization for Beam Hopping and Power Allocation in Dual Satellite Cooperative Transmission NetworksIEEE Systems Journal, 2023
- Dynamic Beam Scheduling of Multibeam Low Earth Orbit Satellites Based on an Enhanced Artificial Bee Colony AlgorithmIEEE Access, 2022
- A Survey on Nongeostationary Satellite Systems: The Communication PerspectiveIEEE Communications Surveys & Tutorials, 2022
- Collaborative Long-Short Term Bandwidth Allocation for Satellite-Terrestrial NetworksIEEE Communications Letters, 2022
- Dynamic Beam Hopping for DVB-S2X GEO Satellite: A DRL-Powered GA ApproachIEEE Communications Letters, 2022
- Integrated Satellite-Terrestrial Networks Toward 6G: Architectures, Applications, and ChallengesIEEE Internet of Things Journal, 2021
- Optimization Method of Dynamic Beam Position for LEO Beam-Hopping Satellite Communication SystemsIEEE Access, 2021
- Beam Illumination Pattern Design in Satellite Networks: Learning and Optimization for Efficient Beam HoppingIEEE Access, 2020
- Multibeam satellite frequency/time duality study and capacity optimizationJournal of Communications and Networks, 2011
- Dynamic Power Allocation for Broadband Multi-Beam Satellite Communication NetworksIEEE Communications Letters, 2011