SZShijin Zhao

Shijin Zhao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Shijin Zhao has a total of 6 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Energy-Efficient AAV Coverage Aware Navigation Under Continuous Dynamic Constraints: An Offline–Online Radio Map-Enhanced DRL Method

    by Shijin Zhao, Fuhui Zhou, Qihui Wu, Feng Shen

    Cellular-connected unmanned aerial vehicles (UAVs) play essential roles across various smart-city applications in low-altitude domains, such as logistics delivery, environmental monitoring, etc. To ensure their reliable deployment, it significantly depends on efficient and intelligent UAV coverage-aware navigation, where deep reinforcement learning (DRL) has emerged as a promising method. However, current DRL-based methods typically simplify UAV flight dynamics by action discretization, which neglects realistic continuous flight dynamic and limits their practical implementation. Moreover, they suffer from the sample inefficiency, since learning the navigation policy requires extensive and costly real-time interactions with the environment. To overcome these two challenges, we propose a novel offline-online radio map-enhanced soft actor-critic (OORM-SAC) framework for the energy-efficient UAV coverage-aware navigation under continuous dynamics constraints. Specifically, our proposed OORM-SAC leverages the classic SAC algorithm to handle large continuous action spaces. It aims to learn continuous steering control to navigate toward the destination while minimizing energy consumption and communication outage. Moreover, to enhance learning efficiency, OORM-SAC adopts a hybrid offline-online learning approach, where the energy-efficient flight pattern is first pre-trained offline and the policy is then refined through online environmental interactions. Furthermore, it incorporates the radio map construction during the online phase to generate diverse virtual training samples, which further accelerates the policy learning. Experimental results demonstrate the effectiveness of OORM-SAC in navigation tasks under continuous dynamic constraints. The OORM-SAC method exhibits superior learning efficiency and generates intelligent trajectories that effectively balance energy consumption and communication requirements.


  • Cognitive Escape Reinforcement Learning for Complex Decision Making

    by Shijin Zhao, Qihui Wu, Fuhui Zhou, Hao Zhang, Yang Huang, Kai-Kuang Ma

    Reinforcement learning (RL) has achieved unprecedented success on making complex decisions encountered in various applications. However, RL has two well-recognized technical challenges, namely, weak scene generalization and local-optima trapping; both lead to degraded performance in terms of navigation accuracy and efficiency. To tackle these issues, a novel RL framework is proposed in this paper, called the cognitive escape reinforcement learning (CERL), which is inspired by the cognition ability inherited in the human beings. Our developed CERL framework consists of two sequential stages: the semantic cognition module (SCM) and the cognitive escape module (CEM), corresponding to the roles played by the perception layer and the decision layer in traditional RL, respectively. The SCM exploits semantic information extracted from the scenes for facilitating the CEM on making complex decisions. Our developed SCM is based on the semantic information extracted from the scenes that is quite capable of adapting to new environments for improving the agent's navigation performance. The CEM proactively identifies the occurrences of local-optima trappings and escapes from such incidences responsively. Extensive experiments have demonstrated the superiority of our proposed RL method, CERL, when compared to the state-of-the-art methods.

  • An Intelligent Multitarget AAV Coverage-Aware Navigation Method for Low-Altitude IoT

    by Shijin Zhao, Fuhui Zhou, Qihui Wu

    Cellular-connected unmanned aerial vehicles (UAVs) act as aerial users to accomplish navigation missions while maintaining a cellular connection, which are of great importance for developing low-altitude Internet of Things (IoT) systems. However, the full aerial cellular coverage is an extreme challenge. To tackle this issue, existing coverage-aware navigation studies focus on designing UAV flight trajectories to avoid weak coverage areas while completing the mission, where considerable efforts of deep reinforcement learning (DRL) based methods have been conducted. However, they lack adaptability to various targets, which limits their practical applicability in real-world scenarios. In this paper, we propose a novel DRL-based multi-target coverage-aware navigation (MTCN) framework, where the UAV learns a navigation policy to minimize the weighted sum of navigation time and the expected communication outage duration. MTCN allows the UAV to efficiently navigate to arbitrary targets without the need of the separate policy training for each target. Furthermore, to improve the sample efficiency of the MTCN, we introduce the radio map-enhanced MTCN (RM-MTCN) framework, which leverages a virtual radio map to generate supplementary navigation training data. RM-MTCN can significantly reduce the need for extensive real-world interactions. Extensive experimental results demonstrate that both MTCN and RM-MTCN achieve generalized navigation behaviors across different targets while avoiding weak communication coverage areas. It is also shown that RM-MTCN exhibits faster learning speed and better performance compared to MTCN, which indicates the effectiveness of leveraging the virtual radio map during the policy training.


  • AAV Visual Navigation in the Large-Scale Outdoor Environment: A Semantic-Map-Based Cognitive Escape Reinforcement Learning Method

    by Shijin Zhao, Fuhui Zhou, Qihui Wu

    The rapid development of the unmanned aerial vehicle (UAV) technology has significantly facilitated its application in the burgeoning Internet of Things (IoT) ecosystem. UAV visual navigation has emerged as a particularly vibrant and crucial research area, holding the potential to greatly enhance automated IoT services. Deep reinforcement learning (DRL), recognized as an effective method for visual navigation, encounters two significant challenges in the complex outdoor environments, i.e., the partial observability and the local optima trapping. In this paper, to address these two challenges, we propose a novel semantic map-based cognitive escape reinforcement learning (SM-CERL) navigation method, which consists of two innovatively designed modules, namely, the semantic mapping module (SMM) and the cognitive escape module (CEM). By deeply exploring the similarity structure of raw images and mapping them to the advanced semantic representations, the SMM constructs a semantic map with rich implications, which provides the UAV with the memory to enhance its understanding of the environment. Meanwhile, the CEM can proactively identify local optima and leverage the rule knowledge to establish efficient escape strategies. The holistic fusion of semantic mapping and cognitive escape mechanisms efficiently enhances the environmental comprehension and prevents UAVs from being trapped in local optima. Extensive experimental results demonstrate that our SM-CERL outperforms the existing classical and state-of-the-art methods in terms of the navigation accuracy and efficiency.