Bing Lu

3D scene perception technology enhances the safety and decision-making ability of autonomous vehicles by accurately acquiring and analyzing stereo information from the environment. Compared to traditional object detection methods, semantic occupancy perception offers greater flexibility in describing 3D scenes with arbitrary shapes and various categories. However, existing methods for semantic occupancy perception face challenges such as poor generalization of depth estimation and inaccurate alignment and fusion of multimodal features. In this paper, a novel multimodal semantic occupancy prediction method, AFOcc, is proposed. AFOcc addresses these challenges by adopting a fusion technique based on feature alignment and an attention mechanism. The method extracts multi-scale features from image and LiDAR data, encodes LiDAR voxel features using sparse convolution, and projects them onto 2D image features for precise alignment. The projection of point clouds onto images is achieved through a feature alignment module. Finally, a learnable fusion module adaptively adjusts the weights of different modal features to enhance the fusion effect. Extensive experiments on the nuScenes-Occupancy dataset demonstrate that AFOcc significantly outperforms state-of-the-art methods in terms of the mIoU metric. Notably, in the bicycle and motorcycle categories, an IoU improvement of more than 40% is achieved. These results illustrate the superior perception and robustness capabilities of AFOcc in complex scenes.

Eco-driving for Fuel Cell Hybrid Electric Vehicles (FCHEVs) through signalized intersections is a coupled problem of speed planning and powertrain control under complex environmental constraints. For global optimality and fast computation, this paper proposes a spatially convex long-term eco-driving approach for FCHEVs on signalized corridors. Considering road slopes, speed limits, and traffic lights, the original ecodriving problem is reformulated as a convex second-order cone programming problem by pre-optimization of the best green light window and convex approximation and convex relaxation of the hybrid powertrain. A powertrain-aware green window planner is first used to determine the optimal passing time windows through signalized intersections. Then the convex eco-driving problem is formulated and finally solved by concurrent optimization and sequential optimization according to whether the speed planning problem and energy management problem are coupled. Results show that the proposed concurrent convex optimization algorithm performs better fuel economy than the sequential optimization algorithm with similar computational time and can reduce motor energy consumption by 4.25% compared to an analytical speed planner. Compared to dynamic programming-based concurrent optimization, the proposed eco-driving method achieves 93.45% fuel economy with only 0.80% computational time

This paper proposes a novel LiDAR inertial odometry (LIO) incorporating vehicle motion characteristics (VMC) tailored for ground vehicles. While most advanced LIO systems are designed for 6 degrees of freedom (DOF) SE(3), which are not ideally suited for ground vehicles, this paper investigates the performance of the vehicle-LiDAR-IMU system by considering the vehicle motion characteristics. The system combines IMU propagation, LiDAR measurements, ground surface measurements and 3 dimension (3D) vehicle kinematic model measurements within an iterated Kalman filter framework. To ensure a robust system bootstrapping, the temporal and spatial parameters of the vehicle-LiDAR-IMU system are calibrated first. Then, in addition to the IMU preintegration and LiDAR measurements, constraints are established between the ground surface and the vehicle pose by calculating the normal of the ground beneath the vehicle in the local point cloud map. Moreover, measurements of the 3D vehicle kinematic model with angular velocity and velocity constraints are adopted. Ground surface is considered in the 3D kinematic model for vehicle angular velocity calculation. The results from simulations and vehicle experiments demonstrate that the proposed method improves accuracy compared with state-of-the-art LiDAR SLAM methods while maintaining a real-time implementation capability for vehicles.

Reliable operation is a key metric for autonomous driving, but the factors related to the environment such as weather and lighting can impair the perception capability of autonomous vehicles. Unsupervised adaptation in foggy circumstances is difficult due to the significant fluctuations in the scene's visibility caused by weather conditions, including smog, fog, and haze. This paper presents an effective network using multi-teacher intermediate domain adaptation (MIFormer), which offers a fresh approach to adapting semantic segmentation for the purpose of handling dense foggy scenes. This network integrates training from an intermediate domain to reduce the domain gap caused by heavy fog weather. To further increase segmentation accuracy, we propose three training strategies: Rare Class Sampling, Multi-teacher knowledge distillation, and Mixup improved by Uncertainty principle. We assess the performance of the suggested model across scenarios involving the adaptation from actual clear-weather scenes to real foggy scenes, as well as from synthetic non-foggy images to real foggy scenes. The results have demonstrated that MIFormer's performance in semantic segmentation with foggy images is better than that with ResNet-38, BDL, CBST, MLST and FogAdapt. In particular, when adhering to the standard configurations and comparing against some mainstream methods, MIFormer achieves a segmentation performance of 69.1 mIoU adapted from GTA to foggy-Cityscapes and 61.8 mIoU adapted from Synthia to foggy-Cityscapes. Additionally, it can even segment rare classes effectively, such as train, bus, and truck.

Multi-object tracking (MOT) in video sequences has gradually become one of the most essential fields in computer vision tasks. As the use of two separate models for feature extraction in two-shot methods precludes them from achieving real-time inference, one-shot methods have been developed to accelerate the running speed by integrating the position estimation of interested targets and extraction of the corresponding appearance embedding features. However, the one-shot MOT accuracy is not as good as that of the two-shot methods. In our study, we design a reinforced one-shot MOT system to further promote the MOT capability. First of all, for the problem of target scale transformation, a receptive field module that applies dilated convolutions to acquire diverse receptive fields is presented. Then, we design an attention mechanism network to extract channel and positional information. Finally, we combine the circle loss with Euclidean distance optimization and cross-entropy loss to enhance the learning of discriminative embeddings. Three public pedestrian tracking datasets are applied to verify the effectiveness and superiority of our presented algorithm. In particular, we achieve 79.0 MOTA, 78.4 IDF1 on the train set of MOT16 with 24.1 FPS inference speed on a NVIDIA GeForce RTX 3090 GPU.

This paper presents a novel interactive motion planning approach for left-turning of autonomous vehicles at uncontrolled intersections. A typical left-turning scenario that consists of agent vehicles and an autonomous ego vehicle is established. The autonomous ego vehicle aims to cross the uncontrolled intersection quickly and safely when meeting with the agent vehicles. A modified obstacle reciprocal collision avoidance (MORCA) prediction model is proposed to predict the trajectory of the agent vehicles with the considerations of longitudinal and lateral interaction-aware behaviors. The parameters of the MORCA prediction model are optimized using particle swarm optimization (PSO) with the inD datasets. Based on MORCA, a partially observable Markov decision process (POMDP) framework is established. The action and state spaces of the POMDP are extended reasonably to satisfy the requirement of real-time application. The simulation results show that the prediction precision is improved by 15% using the MORCA compared with the obstacle reciprocal collision avoidance (ORCA) prediction model. Meanwhile, the results manifest that the proposed MORCA-based POMDP planning method improves 43.8% in commuting efficiency compared with a traditional rule-based method, nearly as good as the vehicle-to-vehicle (V2V) communication is equipped. Note to Practitioners —The motivation of this article is to establish a safe and efficient planning framework for the left-turning of autonomous vehicles at uncontrolled intersections. Rule-based approaches were widely used in the previous studies but they are not optimal. In this paper, the lateral movements of vehicles at the intersection are considered for the first time. A MORCA prediction model is established and optimized to formulate a POMDP planning framework. The CARLA simulation demonstrates the effectiveness of the proposed MORCA-based POMDP planner. In future work, we will migrate the algorithm to a real-world test platform for experimentation.

Accurate trajectory prediction of surrounding vehicles enables lower risk path planning in advance for autonomous vehicles, thus promising the safety of automated driving. A low-risk and high-efficiency path planning approach is proposed for autonomous driving based on the high-performance and practical trajectory prediction method. A long short-term memory (LSTM) network is trained and tested using the highD dataset, and the validated LSTM is used to predict the trajectories of surrounding vehicles combining the information extracted from vehicle-to-vehicle (V2V) technology. A risk assessment and mitigation-based local path planning algorithm is proposed according to the information of predicted trajectories of surrounding vehicles. Two driving scenarios are extracted and reconstructed from the highD dataset for validation and evaluation, i.e., an active lane-change scenario and a longitudinal collision-avoidance scenario. The results illustrate that the risk is mitigated and the driving efficiency is improved with the proposed path planning algorithm comparing to the constant-velocity prediction and the prediction method of the nonlinear input–output (NIO) network, especially when the velocity and trajectory with sudden changes. Note to Practitioners—This article was motivated by the problem of promising the safety decision-making and path planning through accurate environment prediction. There are two main parts included in this article. First, this article proposed one pragmatic approach to predict the environment movement correctly based on the long short-term memory (LSTM) approach. The prediction performance of LSTM was compared with nonlinear input–output (NIO). The results showed that the LSTM approach has a significant advantage in motivation prediction of the surrounded vehicles during path planning. The second part of this article is to make the decision and realize local path planning based on the risk assessment. The potential field-based approach is implemented on the risk assessment based on these accurate predictions. Some primary results demonstrate that the decision-making algorithm performs better under the accurate prediction model. The results also show that the safety and driving efficiency of the ego vehicle were improved by tracking the trajectory, which was planned based on the risk assessment. The only concern for the real-time application is the computation time; in future, we will figure it out how to further reduce the computation time.

The traditional potential field-based path planning is likely to generate unexpected path by strictly following the minimum potential field, especially in the driving scenarios with multiple obstacles closely distributed. A hybrid path planning is proposed to avoid the unsatisfying path generation and to improve the performance of autonomous driving by combining the potential field with the sigmoid curve. The repulsive and attractive potential fields are redesigned by considering the safety and the feasibility. Based on the objective of the shortest path generation, the optimized trajectory is obtained to improve the vehicle stability and driving safety by considering the constraints of collision avoidance and vehicle dynamics. The effectiveness is examined by simulations in multiobstacle dynamic and static scenarios. The simulation results indicate that the proposed method shows better performance on vehicle stability and ride comfortability than that of the traditional potential field-based method in all the examined scenarios during the autonomous driving.

An adaptive potential field is designed to adapt the acceleration/deceleration and mass of the obstacle. The potential fields are established in a transformed road coordinate system to improve the feasibility and robustness. A path planning method is proposed based on the designed adaptive potential field to improve the driving safety and the ride comfort of autonomous vehicles in complex driving scenarios, which including the cut-in, emergency braking, obstacle sudden accelerating during overtaking and the curve road driving scenarios. The effectiveness of the proposed method is validated by simulations with constructed and real data, respectively. The TTCs (Time-to-Collision) and the maximum lateral accelerations are used to evaluate the improvements on safety and ride comfort. The results show that both the driving safety and ride comfort are efficiently improved by using the proposed approach in emergency braking and accelerating scenarios. Meanwhile, the proposed method can be well applied in a curve road driving environment.