Qianqian Zhang

Safe navigation in dynamic unknown environments is challenged by perception uncertainty, moving obstacles, and policy unreliability. Existing DRL-based human-machine collaboration methods rely on autonomous control, fixed authority allocation, or direct action replacement, limiting adaptive coordination under varying risks. To address this problem, this paper proposes an uncertainty-aware constrained-optimization human-machine collaborative reinforcement learning framework. The proposed framework formulates human-machine shared control as a confidence-aware constrained optimization problem for final command generation in dynamic unknown environments. This modeling approach dynamically adjusts the weighting between human guidance and autonomous decision-making based on the reliability of the machine’s strategy. This confidence-aware optimization integrates environmental risk information and policy reliability cues, while kinematic and time-to-collision constraints ensure executable commands within a safe feasible control space. A Bayesian confidence estimator is introduced to quantify the state-dependent reliability of autonomous decisions by integrating environmental risk cues and policy-related features, providing an adaptive confidence weight for human-machine optimization. Within this framework, the proposed human-machine collaborative optimization mechanism is integrated into the Twin Delayed Deep Deterministic Policy Gradient (TD3)-based reinforcement learning framework, where the policy is trained through continuous interaction with the environment to learn autonomous decision-making and collaborative navigation behaviors. The proposed constrained-optimization-formulated shared control framework improves DRL navigation in dynamic unknown environments by executing fused actions in closed-loop interaction and adaptively enhancing stability, safety, and decision robustness.

Distribution system restoration requires coordinated switching decisions to recover de-energized loads subject to network operating constraints. Multi-agent deep reinforcement learning (MADRL) alleviates the scalability challenge of large switching action spaces, yet the decomposition of switching actions among agents remains an important factor. In reconfiguration-based restoration, this decomposition directly affects restoration performance by shaping the feasible restoration paths and supply routes. Existing MADRL-based restoration methods rarely incorporate the network topology into agent organization, leaving potential gains in restoration efficiency unexploited. This paper proposes a topology-driven multi-agent restoration framework that constructs an impedance-weighted graph and applies Leiden community detection to partition the network into agent control regions. Switches crossing community boundaries are assigned to adjacent agents, allowing both agents to operate the corresponding switches. The fault location is used to restrict exploration to fault-relevant agents during training. The resulting multi-agent decision framework is trained using the proposed learning method. Experiments on a practical 82-bus 10~kV distribution network demonstrate the effectiveness of the proposed framework.

Intelligent navigation of mobile robots in unknown environments has become a key enabling technology for service and logistics applications. However, in unstructured scenarios, perception noise and environmental uncertainty can significantly degrade system performance, making it a major challenge to balance safety and efficiency. In this work, we propose a Human–Machine Shared Control method with Model Predictive Control constraints (HMSC). HMSC establishes a confidence-driven human–machine shared control mechanism that maximizes collaborative efficiency by dynamically assessing the reliability of agent decisions to regulate control weights. Simultaneously, the method introduces a composite confidence evaluation model which, by fusing epistemic uncertainty with geometric feasibility from lightweight LiDAR measurements, achieves a robust quantification of policy risk. To ensure safe execution, we develop a multi-trajectory prediction mechanism which, after validating kinematic constraints, minimally intervenes to safely adjust control commands. We conducted Gazebo-based simulation experiments on obstacle avoidance and target navigation using a LiDAR-equipped mobile robot model and validated the rationality of the confidence model. The results demonstrate that the proposed shared control strategy, which combines confidence assessment with deterministic safety boundaries, significantly improves the success rate and robustness of the system in uncertain environments.

Distribution system restoration requires coordinated switching decisions to recover de-energized loads subject to network operating constraints. Multi-agent deep reinforcement learning (MADRL) alleviates the scalability challenge of large switching action spaces, yet the decomposition of switching actions among agents remains an important factor. In reconfiguration-based restoration, this decomposition directly affects restoration performance by shaping the feasible restoration paths and supply routes. Existing MADRL-based restoration methods rarely incorporate the network topology into agent organization, leaving potential gains in restoration efficiency unexploited. This paper proposes a topology-driven multi-agent restoration framework that constructs an impedance-weighted graph and applies Leiden community detection to partition the network into agent control regions. Switches crossing community boundaries are assigned to adjacent agents, allowing both agents to operate the corresponding switches. The fault location is used to restrict exploration to fault-relevant agents during training. The resulting multi-agent decision structure is learned within the proposed framework. Experiments on a practical 82-bus 10~kV distribution network demonstrate the effectiveness of the proposed framework.

As the operation modes and maintenance management requirements of power grids become increasingly complex, the optimization of monthly outage maintenance schedules faces challenges such as difficulty in multi-department coordination, high reliance on manual experience, and insufficient adaptability to dynamic demands. To address these issues, this paper proposes a collaborative optimization method for monthly outage schedules of power grids based on a large language model and reinforcement learning. The method uses the large language model to parse natural language requirements raised in balance meetings, converting requirements such as fixed maintenance dates, duration adjustments, and forbidden maintenance periods into structured constraint parameters. Subsequently, the small reinforcement learning model based on Dueling Deep Q-Network (Dueling DQN) re-optimizes the outage schedule according to the updated environment parameters, achieving a collaborative solution from meeting requirement parsing to schedule generation. Case study results on the IEEE 39-bus system show that the proposed method can integrate the dynamic requirements arising from multiple rounds of balance meetings and perform adaptive optimization of the outage schedule under different constraint conditions. Experimental results on the IEEE 39-bus system demonstrate that the proposed framework effectively optimizes monthly outage schedules. Compared with the initial schedule, the proposed method reduces monthly voltage violations by 31 times and decreases active power loss by 35.88 MWh.

Intelligent navigation of mobile robots in unknown environments has become a key enabling technology for service and logistics applications. However, in unstructured scenarios, perception noise and environmental uncertainty can significantly degrade system performance, making it a major challenge to balance safety and efficiency. In this work, we proposes a Human–Machine Shared Control method with Model Predictive Control constraints (HMSC-MPC). HMSC-MPC establishes a confidence-driven human-machine shared control mechanism that maximizes collaborative efficiency by dynamically assessing the reliability of agent decisions to regulate control weights. Simultaneously, the method introduces a composite confidence evaluation model which, by fusing epistemic uncertainty with geometric feasibility, achieves a robust quantification of policy risk. To ensure safe execution, we develop a multi-trajectory prediction mechanism which, after validating kinematic constraints, minimally intervenes to safely adjust control commands. We conducted obstacle avoidance and target navigation experiments with a mobile robot and validated the rationality of the confidence model. The results demonstrate that the proposed shared control strategy, which combines confidence assessment with deterministic safety boundaries, significantly improves the success rate and robustness of the system in uncertain environments.

To address the inefficient search caused by high-dimensional action spaces in distribution network fault restoration and the safety limitations of purely data-driven decisions in previously unseen scenarios, this paper proposes a knowledge-enhanced reinforcement learning framework, termed DQN-KG-BNN-IQR-Review. More specifically, a knowledge graph(KG) is first used to retrieve high-quality historical cases and provide heuristic action recommendations, thereby narrowing the search space during early training and alleviating inefficient exploration. On this basis, a Bayesian neural network(BNN) is introduced to quantify the epistemic uncertainty of the DQN in real time. By combining these uncertainty estimates with a sliding-window interquartile range(IQR), a dynamic safety boundary is constructed to govern the transition between autonomous model decisions and expert takeover, which improves decision robustness in cognitively unfamiliar states. In addition, a retrospective expert review mechanism is designed to replay successful expert trajectories after failed episodes. By correcting trajectory deviations and steering parameter updates toward more informative directions, this mechanism further enhances generalization. Experimental results show that the proposed framework not only accelerates training convergence, but also achieves an effective balance between restoration efficiency and operational safety in self-healing decision making through the coordinated interaction of its constituent modules.

In clinical EEG-based diagnosis, the acquisition of sufficient and well-labeled data is often hindered by high data collection costs, limited accessibility to subjects, and the inherent difficulty of expert labeling. These constraints result in insufficient data availability, significantly limiting the performance and reliability of traditional machine learning models. To address this challenge, we propose a hybrid human–machine diagnostic framework that integrates deep convolutional generative adversarial networks (DCGANs) and convolutional neural networks (CNNs) for depression detection under incomplete data conditions. The DCGAN module synthesizes realistic EEG samples to augment scarce datasets, while CNNs are employed for feature extraction and classification. A confidence-aware fusion strategy dynamically integrates expert assessments with model predictions, effectively improving diagnostic accuracy in scenarios with limited labeled data. Experimental results on real EEG datasets demonstrate that the proposed approach achieves superior performance, offering a practical solution for intelligent diagnosis in resource-constrained settings.