Throughput maximization using reinforcement learning with state aggregation in satellite revisit scenarios
Authors:
- Felipe PasquevichFPFelipe Pasquevich
- Juan Martin AyardeJAJuan Martin Ayarde
- Yasmín Rayén Mondino LlermanosYLYasmín Rayén Mondino Llermanos
- Graciela Corral BrionesGBGraciela Corral Briones
- Ramin HashemiRHRamin Hashemi
- Risto WichmanRWRisto Wichman
Open Access
- 19 December 2025
- journal article
- Published by Springer Nature in EURASIP Journal on Wireless Communications and Networking
- Vol. 2025 (1)
Abstract
Low Earth Orbit (LEO) satellites are increasingly requiring higher data transfer rates to accommodate the volume of data generated by onboard instruments, such as hyperspectral cameras and synthetic aperture radars. This work proposes a reinforcement learning–based solution to maximize throughput. The central idea is to define a reward function that jointly accounts for packet error rate and modulation scheme. Using the Q-learning algorithm, the agent learns to dynamically select the appropriate modulation. To address the challenge of an infinite state space, an efficient state aggregation strategy is introduced. The evaluation scenario is constructed using real orbital data from the Argentine SAOCOM-1B satellite, represented by Two-Line Elements (TLEs), with a ground station located in Córdoba, Argentina. The channel model assumes free-space path loss (FSPL) and a steerable antenna characterized by its radiation pattern. Four modulation schemes are considered: QPSK, 8-PSK, 16-QAM, and 32-QAM. Numerical results demonstrate that the proposed RL algorithm successfully learns both antenna steering actions and modulation selection, thereby maximizing throughput.Keywords
Funding Information
- Strategic Research Council (RCF project number 339446)
This publication has 13 references indexed in Scilit:
- Adaptive Modulation Using Multi-Objective Reinforcement Learning for LEO Satellites2021
- A Generic Markov Decision Process Model and Reinforcement Learning Method for Scheduling Agile Earth Observation SatellitesIEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020
- Effective Experiences Collection and State Aggregation in Reinforcement LearningIEEE Access, 2019
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Empirical evaluation methods for multiobjective reinforcement learning algorithmsMachine Learning, 2010
- Approximate Q-Learning: An Introduction2010
- On the Limitations of Scalarisation for Multi-objective Reinforcement Learning of Pareto Fronts2008
- Adaptive weighted sum method for multiobjective optimization: a new method for Pareto front generationStructural and Multidisciplinary Optimization, 2005
- Learning Options in Reinforcement Learning2002
- Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming1990