Xin Yuan

This paper investigates a dynamic beam hopping and power allocation problem in rate-splitting multiple access (RSMA) assisted multi-beam satellite system, in which beam services are allocated for terrestrial cells with non-uniform traffic demands and spectrum resource is full-frequency reused among beams for serving multiple ground terminals through RSMA technique. The objective is to jointly optimize beam hopping and power allocation strategy for maximizing the long-term sum rate while simultaneously satisfying delay fairness demand. However, the interference caused by full frequency reuse among beams limits the efficacy of RSMA, and the frequent beam patterns changes make instantaneous channel state information (CSI) detection a challenge. To address these challenges, this paper designs a novel deep reinforcement learning (DRL) based resource allocation framework for discrete-continuous action spaces. To alleviate explosion of action space, each beam is treated as a DRL agent, and a multi-agent deep reinforcement learning method is proposed to learn the beam scheduling policy. In addition, an action mapping mechanism is employed to further manage the DRL power allocation for satisfying the power input constraint arising from limited satellite power budget. Simulation results demonstrate that the proposed method can achieve the highest sum rate of the system while ensuring fairness among non-uniform traffic load cells compared with several baseline methods.

Robots usually appear in the form of robot swarms when they execute complex tasks. Given cost constraints that lead to insufficient computational capabilities in robots, they can offload computation-intensive tasks to mobile edge computing (MEC) servers via satellite-ground network, substantially improving their efficiency in real-world task execution. However, robots' mobility causes their positions relative to MEC servers to change continuously, resulting in dynamically evolving computing offloading environments. To address this issue, this paper aims to ensure the efficiency of multiple robots jointly completing a complex real-world task and at the same time minimize the average completion time of the robot's computing task. We consider a scenario in which multiple robots move to multiple target locations, where the destination of each robot is not assigned in advance. The robots need to move to one of the target locations individually, and at the same time, their own computing tasks can be offloaded to base stations or satellites around them. We conceptualize this problem as a multi-objective optimization framework, which is decoupled into two components: movement control and computation offloading, and propose the DAOMAN algorithm. Simulation results show that the proposed method minimizes the time for robots to reach their respective targets while also reducing the completion time of their computation tasks.

The security of wireless cyber–physical systems (CPSs) faces significant challenges from channel congestion attacks, which disrupt data transmission and threaten system stability. Existing research on attack energy allocation primarily focuses on single-sensor, single-channel scenarios or assumes discrete energy distribution, limiting the development of optimal strategies. To address this, we model the remote state estimation problem in wireless CPSs as a Markov decision process (MDP) and propose an action mapping mechanism (AM). Based on these, we propose two reinforcement learning algorithms to address the continuous attack energy allocation problem in multisensor, multichannel environments. The first algorithm, genetic algorithm-driven model-based action mapping algorithm (GA-MB-AM), is a model-based reinforcement learning approach that uses a genetic algorithm for network initialization. The second algorithm is soft actor–critic-based action mapping algorithm (SAC-AM), which improves performance in dynamic environments, such as systems with energy harvesting. Simulations demonstrate the superiority of our approach over traditional methods.

Autonomous race driving has emerged as a challenging domain for reinforcement learning (RL) applications, requiring high-speed control while adhering to strict safety constraints. Existing RL-based racing methods often struggle to balance performance and safety, with limited adaptability in dynamic racing scenarios with multiple opponent vehicles. The high-dimensional state space and strict safety constraints pose significant challenges for efficient learning. To address these challenges, this paper proposes an integrated RL framework that combines three novel components: (1) a state mapping mechanism that dynamically transforms raw track observations into a consistent representation space; (2) an action mapping technique that rigorously enforces physical traction constraints; and (3) a safe exploration guidance method that combines conservative controllers with RL policies, significantly reducing off-track incidents during training. Extensive experiments conducted in our simulation environment with four test tracks demonstrate the effectiveness of our approach. In time trial scenarios, our method improves lap times by 12–26% and increases the training completion rate from 33.1% to 78.7%. In competitive racing, it achieves a 46–51% higher average speed compared to baseline methods. These results validate the framework’s ability to achieve both high performance and safety in autonomous racing tasks.

Dear Editor, This letter is concerned with the problem of stable high-quality signal transmission of unmanned aerial vehicle (UAV)-assisted multiple-input multiple-output (MIMO) communication system. The particle swarm optimization (PSO) algorithm is used to achieve optimal beamforming and power allocation for this system. Additionally, sensitive particle (SP) and parameter adaptive adjustment are introduced into the traditional PSO algorithm, aiming to improve the performance of the PSO algorithm in dynamic environments with real-time changes in the UAV position. A reinforcement learning (RL)-based approach is proposed to obtain optimal UAV trajectory and adaptive adjustment strategy for PSO parameters, which combine with a specific obstacle avoidance scheme to achieve accurate UAV navigation while satisfying high-quality signal transmission. Simulation experiments show that our scheme provides higher and more stable spectral efficiency as well as more efficient UAV navigation than the currently commonly used scheme with a single RL approach.