Feng Li

Satellites with regenerative payload have become an important way to decrease the end-to-end data transmission latency and improve the resource utilization efficiency in satellite communication systems. However, most of existing works focus on individual uplink or downlink resource allocation without considering the coupling effects of uplink and downlink resource management. In this paper, the coupled uplink-downlink resource allocation with regenerative payloads is analyzed and formulated as a mixed-integer nonlinear programming problem to minimize the completion time of data transmission. To solve the complex problem, a two-stage algorithm is proposed. In the first stage, deep reinforcement learning is adopted to make decisions on the paired user scheduling and channel allocation for uplink and downlink transmission. With the fixed channel allocation decisions, the optimal downlink power allocation can be obtained via successive convex approximation at the second stage. Moreover, the proposed two-stage algorithm can be implemented with low computation complexity. Simulation results show that the proposed algorithm can effectively reduce the completion time of end-to-end data transmission.

In view of the problems existing in the path planning algorithms of unmanned vehicles, such as low search efficiency, slow convergence speed and easy to fall into the local optimal. Based on the characteristics of route planning for unmanned vehicles, this paper introduces Q-Learning into the traditional ant colony algorithm to enhance the learning ability of the algorithm in dynamic environment, so as to improve the overall efficiency of route search. By mapping pheromones into Q values in Q-learning, rapid search in complex environments is realized, and a collection-free path satisfying constraints is quickly found. The results of case analysis show that compared with the traditional ant colony algorithm and the improved ant colony algorithm considering reward and punishment factors, the improved ant colony algorithm based on Q-Learning can effectively reduce the number of iterations, shorten the path optimization time and path length and other performance indicators, and has many advantages in jumping out of the local optimal, improving the global search ability and improving the convergence speed, and has good adaptability and robustness in complex environments. It ensures the safety and stability of unmanned vehicles in complex environments.