Yu Wang

Accurate and feasible target assignment in an urban environment without road networks remains challenging. Existing methods exhibit critical limitations: computational inefficiency preventing real-time decision-making requirements and poor cross-scenario generalization, yielding task-specific policies that lack adaptability. To achieve efficient target assignment in urban adversarial scenarios, we propose an efficient traversable path generation method requiring only binarized images, along with four key constraint models serving as optimization objectives. Moreover, we model this optimization problem as a Markov decision process (MDP) and introduce the generalization sequential proximal policy optimization (GSPPO) algorithm within the reinforcement learning (RL) framework. Specifically, GSPPO integrates an exploration history representation module (EHR) and a neuron-specific plasticity enhancement module (NPE). EHR incorporates exploration history into the policy learning loop, which significantly improves learning efficiency. To mitigate the plasticity loss in neural networks, we propose an NPE module, which boosts the model’s representational capability and generalization across diverse tasks. Experiments demonstrate that our approach reduces planning time by four orders of magnitude compared to the online planning method. Against the benchmark algorithm, it achieves 94.16% higher convergence performance, 33.54% shorter assignment path length, 51.96% lower threat value, and 40.71% faster total time. Our approach supports real-time military reconnaissance and will also facilitate rescue operations in complex cities.

In multi-agent domains, dealing with non-stationary opponents that change behaviors (policies) consistently over time is still a challenging problem, where an agent usually requires the ability to detect the opponent’s policy accurately and adopt the optimal response policy accordingly. Previous works commonly assume that the opponent’s observations and actions during online interactions are known, which can significantly limit their applications, especially in partially observable environments. This paper focuses on efficient policy detecting and reusing techniques against non-stationary opponents without their local information. We propose an algorithm called Bayesian policy reuse with LocAl oBservations (Bayes-Lab) by incorporating variational autoencoders (VAE) into the Bayesian policy reuse (BPR) framework. Following the centralized training with decentralized execution (CTDE) paradigm, we train VAE as an opponent model during the offline phase to extract the latent relationship between the agent’s local observations and the opponent’s local observations. During online execution, the trained opponent models are used to reconstruct the opponent’s local observations, which can be combined with episodic rewards to update the belief about the opponent’s policy. Finally, the agent reuses the best response policy based on the updated belief to improve online performance. We demonstrate that Bayes-Lab outperforms existing state-of-the-art methods in terms of detection accuracy, accumulative rewards, and episodic rewards in a predator–prey scenario. In this experimental environment, Bayes-Lab can achieve about 80% detection accuracy and the highest accumulative rewards, and its performance is less affected by the opponent policy switching interval. When the switching interval is less than 10, its detection accuracy is at least 10% higher than other algorithms.