Yao Chen

Sparse rewards remain a major obstacle to efficient reinforcement learning (RL), particularly in long-horizon goal-reaching tasks where little feedback is available before successful termination. This difficulty is further complicated by time-limited trajectories, for which the absence of observed success indicates incomplete follow-up rather than necessarily a failed attempt. Existing progress-based shaping methods generally do not explicitly model this distinction, which can bias temporal progress estimates and weaken the resulting shaping signal. To address this issue, we propose Censoring-Aware Time-to-Success Potential Shaping (CTPS), a reward-shaping method that treats successful trajectories as observed time-to-event data and time-limited trajectories as right-censored data. CTPS fits a discrete hazard model to estimate the conditional time-to-success distribution, summarizes the resulting survival curve using the restricted mean time-to-success, and converts this estimate into a bounded potential. The learned temporal signal is then incorporated through a potential-based reward difference, with the potential held fixed during each target-update interval. Experiments show that CTPS reduces RMST estimation error by approximately 93% relative to timeout-as-event estimation under heavy censoring, while improving sparse-maze success rates by approximately 157% over NaiveTTS-PBRS. It also preserves state-ranking correlations above 0.98 under the evaluated behavior-policy shifts. These results characterize the temporal-estimation performance of CTPS and its use as a shaping signal in the evaluated sparse-reward tasks.

Adversarial policies provide a practical means of stress-testing reinforcement-learning agents by inducing failure through environment interactions without modifying the deployed victim policy. Learning effective adversarial policies under partial observability is challenging because the attacker must act from incomplete observations without access to victim-side parameters, gradients, or internal states. Existing approaches often rely on instantaneous observability patterns or predefined partitions, which can discard temporal interaction information and provide limited guidance for exploring rarely encountered but potentially vulnerable contexts. We propose CAAP (Context-Adaptive Adversarial Policy), which is a black-box adversarial policy framework that represents recent interaction history with a recurrent Context Encoder, combines multiple action-value experts through a differentiable Soft Policy Router, and encourages early exploration through Context Coverage Exploration using an annealed KNN-based coverage bonus in the learned context space. Context Coverage Exploration is used only during training, leaving the Context Encoder, Soft Policy Router, and action-value experts as the deployed policy. In the primary particle-pursuit benchmark, CAAP achieves the lowest in-domain catch rate among the five internal methods and the lowest macro-average catch rate across eight zero-update transfer conditions. These results support the use of history-dependent Soft Policy Routing with training-only Context Coverage Exploration for black-box adversarial policy learning under partial observability.

High-speed unmanned aerial vehicle (UAV) pursuit-evasion in obstacle-rich environments is a challenging safety-critical decision-making problem, where the pursuer must intercept a maneuvering evader while avoiding obstacles. Existing methods often treat safety as an expected constraint and fail to prevent rare but severe unsafe events, resulting in conservative or aggressive policy. To address this issue, we propose a Risk-Aware Switch-Time Recovery Planning (ReSwitch) framework, which separates task-oriented pursuit from safety-oriented recovery. When flight risk emerges, a value-preserving switch-time planner evaluates candidate switching times through opponent-conditioned hybrid rollouts. The feasible switch time with the largest pursuit value after recovery is selected for execution, allowing the controller to prioritize safety while preserving pursuit effectiveness whenever possible. Experiments demonstrate that ReSwitch improves safety performance over baselines, with less degradation in pursuit effectiveness. The results show that ReSwitch is an effective mechanism for balancing safe flight and task completion.

Distributed coordination algorithms (DCA) carry out information processing processes among a group of networked agents without centralized information fusion. Though it is well known that DCA characterized by an SIA (stochastic, indecomposable, aperiodic) matrix generate consensus asymptotically via synchronous iterations, the dynamics of DCA with asynchronous iterations have not been studied extensively, especially when viewed as stochastic processes. This paper aims to show that for any given irreducible stochastic matrix, even non-SIA, the corresponding DCA lead to consensus successfully via random asynchronous iterations under a wide range of conditions on the transition probability. Particularly, the transition probability is neither required to be independent and identically distributed, nor characterized by a Markov chain.