Guoqing Geng
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Guoqing Geng has a total of 57 co-authors. Here are the top co-authors by number of shared publications:
- Liqin Sun
School of Automotive and Traffic Engineering, Jiangsu University, Zhenjiang, China
6 shared publications - Haobin Jiang
Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China
3 shared publications Chen Duan
Electrical and Computer Engineering, Wayne State University, Detroit, MI, USA
2 shared publicationsDuan Chen
Wayne State University, Detroit, MI, USA
2 shared publicationsHaobin Jiang
School of Automobile and Traffic Engineering, Jiangsu University, Zhenjiang, Jiangsu, China
2 shared publicationsLiqin Sun
2 shared publications
Latest Publications
- 5 November 2026
- Published by Springer Nature in Applied Intelligence
- Vol. 56  (16)
RASA-Drive: from average robustness to long-tail safety diagnosis in scoring-based end-to-end autonomous driving
- 5 November 2026
- Published by Springer Nature in Applied Intelligence
- Vol. 56  (16)
RASA-Drive: from average robustness to long-tail safety diagnosis in scoring-based end-to-end autonomous driving
- 10 March 2026
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
AES-RRT* algorithm: A trajectory planning algorithm for autonomous vehicles in an emergency obstacle avoidance environment
The Rapidly-exploring Random Tree (RRT*) and its variants are widely applied in various domains for solving trajectory planning problems in complex environments. However, in emergency obstacle avoidance scenarios, the trajectories generated by these algorithms often fail to satisfy the dynamic constraints of vehicles, rendering them untrackable by actual automobile systems. To address this limitation, this paper proposes an Adaptive Emergency Sampling-based RRT* (AES-RRT*) algorithm. The proposed algorithm employs a greedy strategy to define a dynamic and stable sampling region, concentrating computational resources on areas with higher probability of containing optimal solutions. By integrating a nonlinear two-degree-of-freedom vehicle model with phase plane analysis, it filters out sampling points that violate dynamic constraints, thereby reducing unnecessary path reconnection optimization for non-essential samples. Furthermore, the algorithm incorporates a variable step-size strategy based on obstacle density around new nodes and distance to the target point, enhancing convergence speed. Finally, Catmull-Rom splines are applied to optimize local paths, improving smoothness and continuity of the final trajectory. Experimental results across three scenarios of varying complexity demonstrate that the AES-RRT* algorithm outperforms RRT, RRT*, and Informed-RRT* in both performance and final planning outcomes. Notably, compared to Informed-RRT*, the proposed algorithm maintains probabilistic completeness and asymptotic optimality while achieving approximately 56.5% faster initial convergence speed, 51.1% faster final convergence speed, and a 74.4% reduction in curvature discontinuity according to path smoothness metrics.
- 10 March 2026
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
AES-RRT* algorithm: A trajectory planning algorithm for autonomous vehicles in an emergency obstacle avoidance environment
The Rapidly-exploring Random Tree (RRT*) and its variants are widely applied in various domains for solving trajectory planning problems in complex environments. However, in emergency obstacle avoidance scenarios, the trajectories generated by these algorithms often fail to satisfy the dynamic constraints of vehicles, rendering them untrackable by actual automobile systems. To address this limitation, this paper proposes an Adaptive Emergency Sampling-based RRT* (AES-RRT*) algorithm. The proposed algorithm employs a greedy strategy to define a dynamic and stable sampling region, concentrating computational resources on areas with higher probability of containing optimal solutions. By integrating a nonlinear two-degree-of-freedom vehicle model with phase plane analysis, it filters out sampling points that violate dynamic constraints, thereby reducing unnecessary path reconnection optimization for non-essential samples. Furthermore, the algorithm incorporates a variable step-size strategy based on obstacle density around new nodes and distance to the target point, enhancing convergence speed. Finally, Catmull-Rom splines are applied to optimize local paths, improving smoothness and continuity of the final trajectory. Experimental results across three scenarios of varying complexity demonstrate that the AES-RRT* algorithm outperforms RRT, RRT*, and Informed-RRT* in both performance and final planning outcomes. Notably, compared to Informed-RRT*, the proposed algorithm maintains probabilistic completeness and asymptotic optimality while achieving approximately 56.5% faster initial convergence speed, 51.1% faster final convergence speed, and a 74.4% reduction in curvature discontinuity according to path smoothness metrics.
- Open Access
Adaptive Sliding Mode Control Based on a Peak-Suppression Extended State Observer for Angle Tracking in Steer-by-Wire Systems
To address the degradation of angle tracking performance in steer-by-wire (SBW) systems caused by external disturbances and parameter uncertainties, this paper proposes a composite control strategy integrating adaptive sliding mode control (ASMC) and a peak-suppression extended state observer (PSESO). Firstly, a novel sliding mode reaching law is designed, which incorporates a dynamic adaptive gain function to achieve real-time adjustment of the control gain. This approach accelerates the reaching speed while effectively mitigating chattering. Secondly, to enhance the disturbance rejection capability of the system, a PSESO is developed to estimate the lumped disturbance in the SBW system in real time. By dynamically adjusting the observer bandwidth, the peak phenomenon in state estimation is suppressed, thereby avoiding saturation of the control signal. The disturbance estimate from the PSESO is then fed forward as a compensation term into the adaptive sliding mode (ASM) controller, forming a composite ASMC+PSESO controller that enables active compensation and suppression of disturbances. Finally, the proposed composite control strategy is validated through both simulations and experiments. Experimental results demonstrate that under sinusoidal signal tracking conditions, the proposed method reduces the maximum tracking error, the mean absolute error, and the integral absolute error by 64.4%, 74.2%, and 73.1%, respectively, compared to traditional sliding mode control (TSMC). These results fully underscore its superiority in angle tracking control and disturbance rejection for SBW systems.
Open AccessAdaptive Sliding Mode Control Based on a Peak-Suppression Extended State Observer for Angle Tracking in Steer-by-Wire Systems
To address the degradation of angle tracking performance in steer-by-wire (SBW) systems caused by external disturbances and parameter uncertainties, this paper proposes a composite control strategy integrating adaptive sliding mode control (ASMC) and a peak-suppression extended state observer (PSESO). Firstly, a novel sliding mode reaching law is designed, which incorporates a dynamic adaptive gain function to achieve real-time adjustment of the control gain. This approach accelerates the reaching speed while effectively mitigating chattering. Secondly, to enhance the disturbance rejection capability of the system, a PSESO is developed to estimate the lumped disturbance in the SBW system in real time. By dynamically adjusting the observer bandwidth, the peak phenomenon in state estimation is suppressed, thereby avoiding saturation of the control signal. The disturbance estimate from the PSESO is then fed forward as a compensation term into the adaptive sliding mode (ASM) controller, forming a composite ASMC+PSESO controller that enables active compensation and suppression of disturbances. Finally, the proposed composite control strategy is validated through both simulations and experiments. Experimental results demonstrate that under sinusoidal signal tracking conditions, the proposed method reduces the maximum tracking error, the mean absolute error, and the integral absolute error by 64.4%, 74.2%, and 73.1%, respectively, compared to traditional sliding mode control (TSMC). These results fully underscore its superiority in angle tracking control and disturbance rejection for SBW systems.
- Open Access
- 12 February 2026
- Published by MDPI AG in World Electric Vehicle Journal
- Vol. 17  (2) , 92
Design and Performance Analysis of a Vehicle Vibration Energy Harvester Based on Piezoelectric Technology with Nonlinear Magnetic Coupling
To address the waste of mechanical energy from suspension vibrations during vehicle operation, this study proposes a vehicle suspension vibration energy harvester based on the piezoelectric effect and nonlinear magnetic coupling. It aims to recover the mechanical energy generated by suspension vibrations in the course of vehicle operation. The device adopts a multi-cantilever beam array structure. Permanent magnets are symmetrically arranged on the free ends of cantilevers and suspension springs, which enables non-contact excitation and system frequency regulation. It converts mechanical energy into electrical energy by virtue of the direct piezoelectric effect. A finite element simulation model was developed in the study. A dedicated vibration test platform was also constructed. Experimental results show the following performance: Under the operating conditions of 16.75 Hz excitation frequency and 10 kΩ load resistance, a single cantilever beam can generate a peak voltage of 9.59 V. Its maximum output power reaches 7.67 mW. Under simulated Class D road conditions and at a vehicle speed of 90 km/h, the array made up of eight cantilever beams delivers a total output power of 414.37 mW. This study provides a viable technical solution for vehicle suspension vibration energy recovery. It promotes the full utilization of wasted energy, and it is of great significance for advancing sustainable development in the transportation sector.
- 12 February 2026
- Published by MDPI AG in World Electric Vehicle Journal
- Vol. 17  (2) , 92
Open AccessDesign and Performance Analysis of a Vehicle Vibration Energy Harvester Based on Piezoelectric Technology with Nonlinear Magnetic Coupling
To address the waste of mechanical energy from suspension vibrations during vehicle operation, this study proposes a vehicle suspension vibration energy harvester based on the piezoelectric effect and nonlinear magnetic coupling. It aims to recover the mechanical energy generated by suspension vibrations in the course of vehicle operation. The device adopts a multi-cantilever beam array structure. Permanent magnets are symmetrically arranged on the free ends of cantilevers and suspension springs, which enables non-contact excitation and system frequency regulation. It converts mechanical energy into electrical energy by virtue of the direct piezoelectric effect. A finite element simulation model was developed in the study. A dedicated vibration test platform was also constructed. Experimental results show the following performance: Under the operating conditions of 16.75 Hz excitation frequency and 10 kΩ load resistance, a single cantilever beam can generate a peak voltage of 9.59 V. Its maximum output power reaches 7.67 mW. Under simulated Class D road conditions and at a vehicle speed of 90 km/h, the array made up of eight cantilever beams delivers a total output power of 414.37 mW. This study provides a viable technical solution for vehicle suspension vibration energy recovery. It promotes the full utilization of wasted energy, and it is of great significance for advancing sustainable development in the transportation sector.
- Open Access
- 25 August 2025
- Published by MDPI AG in World Electric Vehicle Journal
- Vol. 16  (9) , 483
YOLOv5s-F: An Improved Algorithm for Real-Time Monitoring of Small Targets on Highways
To address the challenges of real-time monitoring via highway vehicle-mounted cameras—specifically, the difficulty in detecting distant pedestrians and vehicles in real time—this study proposes an enhanced object detection algorithm, YOLOv5s-F. Firstly, the FasterNet network structure is adopted to improve the model’s runtime speed. Secondly, the attention mechanism BRA, which is derived from the Transformer algorithm, and a 160 × 160 small-object detection layer are introduced to enhance small target detection performance. Thirdly, the improved upsampling operator CARAFE is incorporated to boost the localization and classification accuracy of small objects. Finally, Focal EIoU is employed as the localization loss function to accelerate model training convergence. Quantitative experiments on high-speed sequences show that Focal EIoU reduces bounding box jitter by 42.9% and improves tracking stability (consecutive frame overlap) by 11.4% compared to CIoU, while accelerating convergence by 17.6%. Results show that compared with the YOLOv5s baseline network, the proposed algorithm reduces computational complexity and parameter count by 10.1% and 24.6%, respectively, while increasing detection speed and accuracy by 15.4% and 2.1%. Transfer learning experiments on the VisDrone2019 and Highway-100k dataset demonstrate that the algorithm outperforms YOLOv5s in average precision across all target categories. On NVIDIA Jetson Xavier NX, YOLOv5s-F achieves 32 FPS after quantization, meeting the real-time requirements of in-vehicle monitoring. The YOLOv5s-F algorithm not only meets the real-time detection and accuracy requirements for small objects but also exhibits strong generalization capabilities. This study clarifies core challenges in highway small-target detection and achieves accuracy–speed improvements via three key innovations, with all experiments being reproducible. If any researchers need the code and dataset of this study, they can consult the author through email.
- 25 August 2025
- Published by MDPI AG in World Electric Vehicle Journal
- Vol. 16  (9) , 483
Open AccessYOLOv5s-F: An Improved Algorithm for Real-Time Monitoring of Small Targets on Highways
To address the challenges of real-time monitoring via highway vehicle-mounted cameras—specifically, the difficulty in detecting distant pedestrians and vehicles in real time—this study proposes an enhanced object detection algorithm, YOLOv5s-F. Firstly, the FasterNet network structure is adopted to improve the model’s runtime speed. Secondly, the attention mechanism BRA, which is derived from the Transformer algorithm, and a 160 × 160 small-object detection layer are introduced to enhance small target detection performance. Thirdly, the improved upsampling operator CARAFE is incorporated to boost the localization and classification accuracy of small objects. Finally, Focal EIoU is employed as the localization loss function to accelerate model training convergence. Quantitative experiments on high-speed sequences show that Focal EIoU reduces bounding box jitter by 42.9% and improves tracking stability (consecutive frame overlap) by 11.4% compared to CIoU, while accelerating convergence by 17.6%. Results show that compared with the YOLOv5s baseline network, the proposed algorithm reduces computational complexity and parameter count by 10.1% and 24.6%, respectively, while increasing detection speed and accuracy by 15.4% and 2.1%. Transfer learning experiments on the VisDrone2019 and Highway-100k dataset demonstrate that the algorithm outperforms YOLOv5s in average precision across all target categories. On NVIDIA Jetson Xavier NX, YOLOv5s-F achieves 32 FPS after quantization, meeting the real-time requirements of in-vehicle monitoring. The YOLOv5s-F algorithm not only meets the real-time detection and accuracy requirements for small objects but also exhibits strong generalization capabilities. This study clarifies core challenges in highway small-target detection and achieves accuracy–speed improvements via three key innovations, with all experiments being reproducible. If any researchers need the code and dataset of this study, they can consult the author through email.
- Open Access
- 2 April 2025
- Published by MDPI AG in World Electric Vehicle Journal
- Vol. 16  (4)
Enhanced Isolation Forest-Based Algorithm for Unsupervised Anomaly Detection in Lidar SLAM Localization
Lidar SLAM (simultaneous localization and mapping) systems provide vehicles with high-precision maps and localization for environmental perception. However, sensor noise and dynamic changes can lead to the localization drift or localization failure of the SLAM system. Identifying such anomalies currently relies on post-trajectory analysis with subjective parameter thresholds. To address this issue, we propose an unsupervised real-time localization anomaly detection model based on the isolation forest algorithm. We first determined the necessity of variable research through variable correlation analysis. Then, we enhanced the scoring mechanism of the isolation forest by introducing a path-weighting method, improving sensitivity to complex variables and anomalies. Finally, to further increase the model’s reliability, we employed an adaptive OTSU (Otsu’s method) algorithm for automatic score classification. Experimental results show that our proposed model can effectively detect positioning anomalies by determining variable thresholds in four scenarios of the KITTI dataset. The results show excellent real-time performance, with an average running time of about 0.02 s, which is shorter than the time required to process a single data frame. Using the mean, RMSE, and standard deviation as evaluation metrics, data comparisons confirmed the algorithm’s accuracy. Compared with several SOTA (state-of-the-art) algorithms and ablation studies, our algorithm also showed higher sensitivity.
- 2 April 2025
- Published by MDPI AG in World Electric Vehicle Journal
- Vol. 16  (4)
Open AccessEnhanced Isolation Forest-Based Algorithm for Unsupervised Anomaly Detection in Lidar SLAM Localization
Lidar SLAM (simultaneous localization and mapping) systems provide vehicles with high-precision maps and localization for environmental perception. However, sensor noise and dynamic changes can lead to the localization drift or localization failure of the SLAM system. Identifying such anomalies currently relies on post-trajectory analysis with subjective parameter thresholds. To address this issue, we propose an unsupervised real-time localization anomaly detection model based on the isolation forest algorithm. We first determined the necessity of variable research through variable correlation analysis. Then, we enhanced the scoring mechanism of the isolation forest by introducing a path-weighting method, improving sensitivity to complex variables and anomalies. Finally, to further increase the model’s reliability, we employed an adaptive OTSU (Otsu’s method) algorithm for automatic score classification. Experimental results show that our proposed model can effectively detect positioning anomalies by determining variable thresholds in four scenarios of the KITTI dataset. The results show excellent real-time performance, with an average running time of about 0.02 s, which is shorter than the time required to process a single data frame. Using the mean, RMSE, and standard deviation as evaluation metrics, data comparisons confirmed the algorithm’s accuracy. Compared with several SOTA (state-of-the-art) algorithms and ablation studies, our algorithm also showed higher sensitivity.
- Open Access
- 9 February 2025
- Published by MDPI AG in Sustainability
- Vol. 17  (4) , 1412
Approximate Globally Optimal Energy Management Strategy for Fuel Cell Hybrid Mining Trucks Based on Rule-Interposing Balance Cost Minimization
Fuel cell hybrid vehicles offer significant potential in heavy-duty transportation due to their high efficiency, extended range, and zero emissions, making them a key enabler of sustainable transportation. To enhance the energy consumption economy and lifecycle economy of fuel cell hybrid mining trucks (FCHMTs) while reducing total operating costs and promoting environmental sustainability, this paper proposes an approximate globally optimal energy management strategy (EMS) based on a rule-interposing balance cost minimization strategy (AGO-BCMS). First, an FCHMT power system model is established, including degradation models for the fuel cell and battery. Then, the global optimality of dynamic programming (DP) is utilized to extract the fuel cell output characteristics under different battery states and vehicle power demands. Subsequently, optimal rules are designed and embedded into the cost minimization optimization model to plan the fuel cell output range under actual driving conditions. Simultaneously, dynamic threshold updates are performed based on vehicle driving condition phase recognition. Finally, energy distribution optimization is calculated using sequential quadratic programming (SQP). This strategy not only improves the economic viability of FCHMTs but also contributes to the broader goals of advancing sustainable transportation solutions. The proposed strategy was validated under both single round-trip and continuous operational conditions. Simulation results show that, under single round-trip conditions, the proposed strategy reduces the total operational cost by 3.13%, 4.09%, and 10.90% compared to balance cost-minimization strategies (BCMS), equivalent consumption minimization strategy (ECMS), and rule-based strategies, respectively. Under continuous operational conditions, the total cost is reduced by 3.61%, 6.63%, and 15.90%, respectively.
- 9 February 2025
- Published by MDPI AG in Sustainability
- Vol. 17  (4) , 1412
Open AccessApproximate Globally Optimal Energy Management Strategy for Fuel Cell Hybrid Mining Trucks Based on Rule-Interposing Balance Cost Minimization
Fuel cell hybrid vehicles offer significant potential in heavy-duty transportation due to their high efficiency, extended range, and zero emissions, making them a key enabler of sustainable transportation. To enhance the energy consumption economy and lifecycle economy of fuel cell hybrid mining trucks (FCHMTs) while reducing total operating costs and promoting environmental sustainability, this paper proposes an approximate globally optimal energy management strategy (EMS) based on a rule-interposing balance cost minimization strategy (AGO-BCMS). First, an FCHMT power system model is established, including degradation models for the fuel cell and battery. Then, the global optimality of dynamic programming (DP) is utilized to extract the fuel cell output characteristics under different battery states and vehicle power demands. Subsequently, optimal rules are designed and embedded into the cost minimization optimization model to plan the fuel cell output range under actual driving conditions. Simultaneously, dynamic threshold updates are performed based on vehicle driving condition phase recognition. Finally, energy distribution optimization is calculated using sequential quadratic programming (SQP). This strategy not only improves the economic viability of FCHMTs but also contributes to the broader goals of advancing sustainable transportation solutions. The proposed strategy was validated under both single round-trip and continuous operational conditions. Simulation results show that, under single round-trip conditions, the proposed strategy reduces the total operational cost by 3.13%, 4.09%, and 10.90% compared to balance cost-minimization strategies (BCMS), equivalent consumption minimization strategy (ECMS), and rule-based strategies, respectively. Under continuous operational conditions, the total cost is reduced by 3.61%, 6.63%, and 15.90%, respectively.
- 30 January 2025
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
- Vol. 240  (1) , 91-104
Optimal control of clamping force in EMB system based on nonlinear disturbance observer
This paper investigates the optimization control of clamping force in electronic-mechanical braking (EMB) systems, which are characterized by significant nonlinearities. To address the impact of factors such as gaps, wear, friction, and disturbances on the clamping force output accuracy within the brake actuator, an improved sliding mode control strategy (ISMC) embedded with a nonlinear disturbance observer (NDO) is proposed. The control strategy targets the clamping force and comprehensively accounts for the nonlinear aspects of the EMB system by modeling them as composite disturbances. A cascaded NDO is used to predict and compensate for the total disturbances in real-time, thereby improving control precision. Furthermore, the sliding mode control rate is enhanced by replacing the traditional switching function with a saturation function to mitigate chattering effects. Simulation and experimental results demonstrate that the proposed EMB controller can rapidly and accurately align the output clamping force with the target value, without compromising system stability. It significantly suppresses chattering, adapts to road surfaces with varying friction coefficients, and improves response speed by 44.2%, while reducing overshoot by an average of 79.67% across various braking conditions. Additionally, the controller is capable of rapidly and accurately adjusting the target value in response to sudden wheel load changes, ensuring stability and reliability during the braking process and reducing the vehicle’s braking distance by an average of 7.61%. In long-duration braking tests, the disturbance compensation mechanism allows the controller to estimate and compensate for output errors in real-time, minimizing the risk of performance degradation due to prolonged braking. In conclusion, the EMB controller designed in this study significantly enhances the braking system’s response speed, stability, and safety under dynamic and complex conditions.
- 30 January 2025
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
- Vol. 240  (1) , 91-104
Optimal control of clamping force in EMB system based on nonlinear disturbance observer
This paper investigates the optimization control of clamping force in electronic-mechanical braking (EMB) systems, which are characterized by significant nonlinearities. To address the impact of factors such as gaps, wear, friction, and disturbances on the clamping force output accuracy within the brake actuator, an improved sliding mode control strategy (ISMC) embedded with a nonlinear disturbance observer (NDO) is proposed. The control strategy targets the clamping force and comprehensively accounts for the nonlinear aspects of the EMB system by modeling them as composite disturbances. A cascaded NDO is used to predict and compensate for the total disturbances in real-time, thereby improving control precision. Furthermore, the sliding mode control rate is enhanced by replacing the traditional switching function with a saturation function to mitigate chattering effects. Simulation and experimental results demonstrate that the proposed EMB controller can rapidly and accurately align the output clamping force with the target value, without compromising system stability. It significantly suppresses chattering, adapts to road surfaces with varying friction coefficients, and improves response speed by 44.2%, while reducing overshoot by an average of 79.67% across various braking conditions. Additionally, the controller is capable of rapidly and accurately adjusting the target value in response to sudden wheel load changes, ensuring stability and reliability during the braking process and reducing the vehicle’s braking distance by an average of 7.61%. In long-duration braking tests, the disturbance compensation mechanism allows the controller to estimate and compensate for output errors in real-time, minimizing the risk of performance degradation due to prolonged braking. In conclusion, the EMB controller designed in this study significantly enhances the braking system’s response speed, stability, and safety under dynamic and complex conditions.
- 18 May 2024
- Published by Springer Nature in International Journal of Automotive Technology
- Vol. 25  (6) , 1415-1429
A Study on Lateral Stability Control of Distributed Drive Electric Vehicle Based on Fuzzy Adaptive Sliding Mode Control
- 18 May 2024
- Published by Springer Nature in International Journal of Automotive Technology
- Vol. 25  (6) , 1415-1429
A Study on Lateral Stability Control of Distributed Drive Electric Vehicle Based on Fuzzy Adaptive Sliding Mode Control
- 29 January 2024
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
- Vol. 239  (5) , 1505-1516
A decision-making of autonomous driving method based on DDPG with pretraining
Present the DDPGwP (DDPG with Pretraining) model, grounded in the framework of deep reinforcement learning, designed for autonomous driving decision-making. The model incorporates imitation learning by utilizing expert experience for supervised learning during initial training and weight preservation. A novel loss function is devised, enabling the expert experience to jointly guide the Actor network’s update alongside the Critic network while also participating in the Critic network’s updates. This approach allows imitation learning to dominate the early stages of training, with reinforcement learning taking the lead in later stages. Employing experience replay buffer separation techniques, we categorize and store collected superior, ordinary, and expert experiences. We select sensor inputs from the TORCS (The Open Racing Car Simulator) simulation platform and conduct experimental validation, comparing the results with the original DDPG, A2C, and PPO algorithms. Experimental outcomes reveal that incorporating imitation learning significantly accelerates early-stage training, reduces blind trial-and-error during initial exploration, and enhances algorithm stability and safety. The experience replay buffer separation technique improves sampling efficiency and mitigates algorithm overfitting. In addition to expediting algorithm training rates, our approach enables the simulated vehicle to learn superior strategies, garnering higher reward values. This demonstrates the superior stability, safety, and policy-making capabilities of the proposed algorithm, as well as accelerated network convergence.
- 29 January 2024
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
- Vol. 239  (5) , 1505-1516
A decision-making of autonomous driving method based on DDPG with pretraining
Present the DDPGwP (DDPG with Pretraining) model, grounded in the framework of deep reinforcement learning, designed for autonomous driving decision-making. The model incorporates imitation learning by utilizing expert experience for supervised learning during initial training and weight preservation. A novel loss function is devised, enabling the expert experience to jointly guide the Actor network’s update alongside the Critic network while also participating in the Critic network’s updates. This approach allows imitation learning to dominate the early stages of training, with reinforcement learning taking the lead in later stages. Employing experience replay buffer separation techniques, we categorize and store collected superior, ordinary, and expert experiences. We select sensor inputs from the TORCS (The Open Racing Car Simulator) simulation platform and conduct experimental validation, comparing the results with the original DDPG, A2C, and PPO algorithms. Experimental outcomes reveal that incorporating imitation learning significantly accelerates early-stage training, reduces blind trial-and-error during initial exploration, and enhances algorithm stability and safety. The experience replay buffer separation technique improves sampling efficiency and mitigates algorithm overfitting. In addition to expediting algorithm training rates, our approach enables the simulated vehicle to learn superior strategies, garnering higher reward values. This demonstrates the superior stability, safety, and policy-making capabilities of the proposed algorithm, as well as accelerated network convergence.