Bo Hu

Decision-making and control are the core functionalities of high-level autonomous driving systems. Existing mainstream research, including modular and end-to-end paradigms, typically employ conservative strategies that compromise driving efficiency. However, driving efficiency constitutes a critical constraint on the transition of autonomous vehicles from mere operability to practical utility. Autonomous overtaking systems serve as a typical means to improve driving efficiency. Nevertheless, in stochastic and uncertain traffic scenarios, achieving safe and efficient continuous autonomous overtaking remains a significant challenge. In this context, this paper proposes a decision-making and control framework based on MeUAL to achieve the optimal trade-off between overtaking risk and efficiency. First, at the decision-making layer, a safe reinforcement learning method based on Uncertainty-aware Augmented Lagrangian (UAL) is developed to provide global overtaking guidance. Subsequently, the motion planning and control layer based on Model Predictive Control (MPC) closely tracks the UAL-generated guidance, while preserving the safety and constraint guarantees inherent to traditional MPC. Finally, a Policy Switching Mechanism (PSM) triggered by the safety epistemic uncertainty threshold is designed for the MeUAL-driven autonomous overtaking system. Experimental results demonstrate that MeUAL outperforms baseline algorithms with respect to reward-cost balance, sample efficiency, and learning stability. Moreover, in various test scenarios that are distinct from the training distribution, MeUAL-PSM exhibits strong robustness and interpretable overtaking maneuvers through flexible policy switching.

This paper presents an energy management strategy (EMS) for plug-in fuel cell hybrid electric vehicles (PHEVs) aimed at minimizing both fuel consumption and power source degradation. Firstly, an Internet-distributed vehicle-in-the-loop (ID-VIL) simulation system is introduced, in which a cloud server, an on-road driving test vehicle, a fuel cell stack, and a power battery pack are coupled together through the Internet to evaluate energy management's influence on power source degradation. Secondly, a two-layer structure Internet-distributed energy management strategy (ID-EMS) is presented. The cloud-deployed top-layer optimizes the power split of PHEVs globally. Meanwhile, the bottom-layer onboard uses the top-layer's result as a reference to guide its real-time local optimization. Thus, real-time approximate global optimization is achieved. Lastly, based on the ID-VIL system, ID-EMS is tested in a 30-minute on-road driving experiment. Compared to the dynamic programming-based EMS (DP-EMS) and rule-based EMS (RB-EMS), the proposed ID-EMS can globally optimize fuel consumption and degradation of power sources in a real-time environment.

Motion planning is the core component of autonomous driving technology. Traditional rule- and optimization-based methods face limitations when handling long-tail scenarios and complex traffic dynamics. While learning-based approaches offer potential solutions, they often compromise on trajectory safety and interpretability. In this context, this article introduces a transformer optimized planner (TOP), which combines generative artificial intelligence (AI) with optimization methods for autonomous driving on-ramping merging task. Specifically, TOP treats motion planning tasks as an offline reinforcement learning (RL) problem and employs the powerful sequence modeling capabilities of transformers, typically used in natural language processing (NLP), to derive an initial planning trajectory that maximizes utility from a fixed, previously collected dataset. Additionally, TOP facilitates real-time planning by utilizing efficient tokenization and temporal abstraction to compress input sequences into high-level latent actions. On this basis, optimization methods are employed to refine and safely adjust the transformer-generated trajectory, ensuring it adheres to safety requirements and vehicle kinematic constraints. To evaluate the effectiveness of the proposed algorithm, we performed simulations and conducted experiments using the Apollo industrial control unit, comparing the performance of TOP with the transformer- and optimization-based planner in an on-ramp merging scenario. It is found that TOP guarantees that the motion planning decisions are made within tight latency constraints (such as 100 ms target) even in complex scenarios, demonstrating 33 times shorter online execution time compared to the transformer-based planner and was nearly three times faster than the optimization-based planner while ensuring safety and performance.

Autonomous vehicles need to continuously analyse the driving context and establish a comprehensive understanding of the dynamic traffic environment. To ensure the safety and efficiency of their operations, it would be beneficial to have accurate predictions of surrounding vehicles’ future trajectories. AVs can adjust their motions proactively to improve road safety and comfort with such information. This paper proposes a novel approach to predict the future trajectories of interacting vehicles, through a model of potential spatial-temporal interactions. A unique kernel function that emphasises risk-awareness was developed to extract spatial dependencies. The established model was trained and evaluated with the publicly available Highway Drone Dataset and Intersection Drone Dataset. The performance of the developed model was assessed with eight state-of-the-art methods. An ablation study and safety analysis were also conducted to evaluate the proposed risk-awareness kernel function. Results show that the proposed model’s inference speed is over eight times faster than the commonly used LSTM-based models. It also achieves an improvement of over 8% in prediction accuracy when compared with the state-of-the-art model.

The intricate interactions with other road users and the diversity of traffic environments create a challenging decision-making task for autonomous driving systems. While offline learning solutions are renowned for their high execution efficiency and the ability to approximate the optimal policy across the entire state space, they are often unsafe and fragile when encountering untrained states. Conversely, online planning methods possess the capacity to thoroughly assess how current decisions influence future outcomes online, and therefore have better generalization. Nevertheless, these approaches face difficulties in terms of execution efficiency and are susceptible to becoming stuck in locally optimal solutions. In this context, this paper proposes an Integrated Planning and Learning (IPL) algorithm based on the reinforcement learning framework for speed and lane change decision-making on highways. Specifically, at each decision time step, this method utilizes the offline learned model to guide an online Monte Carlo Tree Search (MCTS) algorithm for heuristic search, aiming to formulate a forward-looking policy. The experimental results show that the IPL algorithm performs better generalization when faced with unknown scenarios, and its asymptotic performance is better than other benchmark algorithms. In addition, in contrast to the MCTS-based online planning method, the IPL algorithm enhances execution efficiency and comes closer to achieving global optimality.

Considering the simulation-to-real gap and the fact that the data-driven learning methods are often suboptimal, an effective offline-to-online learning paradigm that can further improve the initialized offline policy in a real environment is necessary. However, for most industrial applications, this two-stage learning paradigm is often challenging to implement, as the initialized policy frequently deviates from the optimal improvement during fine-tuning. To address this, an effective offline-to-online reinforcement learning (RL) based training framework that bridges offline initialization and online fine-tuning will be proposed in this work. In contrast to most of RL algorithms, which evaluate unseen actions from the latest policy and extract policies only corresponding to the maximum value function, the proposed method uses two implicit constraints, namely, quantile regression and advantage weighted regression. In order to assess the efficacy of the proposed algorithm with real controllers, a hardware-in-the-loop test is conducted. The findings demonstrate that by leveraging the generalization capability of the quantile regression to estimate the value of the best available state-action pair and derive the policy using the advantage-weighted form of behavioral cloning to favor actions that receive higher advantage, the proposed data-driven RL can not only learn effectively from a static offline dataset, but also exhibit a robust policy improvement during the subsequent fine-tuning process.

Reinforcement learning (RL) has the capability to discover optimal interactions with the surrounding environment, with the advantage that nearly all required computations can be performed offline. Nevertheless, the lack of explainability for RL-based solutions may prevent their large-scale application in industrial autonomous vehicle tasks. Furthermore, the RL method tends to be unsafe and brittle to scenarios not encountered in training. Conversely, the optimization-based method offers a substantial level of explainability, and through the explicit inclusion of safety constraints, it can guarantee the system’s safety. In this context, building upon RL framework, a fusion algorithm that combines the advantage of the RL-based scheme and optimization-based scheme is proposed. Specifically, unlike traditional RL-based solutions which directly executes from perception to control using only neural network maps, this work introduces a mechanism of uncertainty-aware interval prediction to compute the set of states that can be reached over the planning time horizon. On this basis, a robust control framework is presented, which guarantees system safety while considering the worst-case performance scenarios. To validate the proposed algorithm, the task of an autonomous vehicle merging on to a highway from an on-ramp is simulated in SUMO. The results show that the proposed motion planning and control method combines the advantages of RL and optimization-based methods and achieves balanced performance in smoothness, computational efficiency, explainability and robustness.