Bocheng Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Bocheng Liu has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Bo Hu
Key Laboratory of Advanced Manufacturing Technology for Automobile Parts, Ministry of Education, Chongqing University of Technology, Chongqing, China
2 shared publications - Sunan Zhang
School of Mechanical Engineering, Southeast University, Nanjing, China
2 shared publications Junxiong Chai
School of Electronics and Information Engineering, Southwest University, China
1 shared publicationManying Feng
School of Electronics and Information Engineering, Southwest University, China
1 shared publicationQianfeng Tang
School of Electronics and Information Engineering, Southwest University, China
1 shared publication- Tingting Song
Department of Electrical and Electronic Engineering, The University of Melbourne, VIC, Australia
1 shared publication
Latest Publications
- 6 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 71 (10) , 12869-12878
A Data-Driven Reinforcement Learning Based Energy Management Strategy via Bridging Offline Initialization and Online Fine-Tuning for a Hybrid Electric Vehicle
Considering the simulation-to-real gap and the fact that the data-driven learning methods are often suboptimal, an effective offline-to-online learning paradigm that can further improve the initialized offline policy in a real environment is necessary. However, for most industrial applications, this two-stage learning paradigm is often challenging to implement, as the initialized policy frequently deviates from the optimal improvement during fine-tuning. To address this, an effective offline-to-online reinforcement learning (RL) based training framework that bridges offline initialization and online fine-tuning will be proposed in this work. In contrast to most of RL algorithms, which evaluate unseen actions from the latest policy and extract policies only corresponding to the maximum value function, the proposed method uses two implicit constraints, namely, quantile regression and advantage weighted regression. In order to assess the efficacy of the proposed algorithm with real controllers, a hardware-in-the-loop test is conducted. The findings demonstrate that by leveraging the generalization capability of the quantile regression to estimate the value of the best available state-action pair and derive the policy using the advantage-weighted form of behavioral cloning to favor actions that receive higher advantage, the proposed data-driven RL can not only learn effectively from a static offline dataset, but also exhibit a robust policy improvement during the subsequent fine-tuning process.
- 6 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 71 (10) , 12869-12878
A Data-Driven Reinforcement Learning Based Energy Management Strategy via Bridging Offline Initialization and Online Fine-Tuning for a Hybrid Electric Vehicle
Considering the simulation-to-real gap and the fact that the data-driven learning methods are often suboptimal, an effective offline-to-online learning paradigm that can further improve the initialized offline policy in a real environment is necessary. However, for most industrial applications, this two-stage learning paradigm is often challenging to implement, as the initialized policy frequently deviates from the optimal improvement during fine-tuning. To address this, an effective offline-to-online reinforcement learning (RL) based training framework that bridges offline initialization and online fine-tuning will be proposed in this work. In contrast to most of RL algorithms, which evaluate unseen actions from the latest policy and extract policies only corresponding to the maximum value function, the proposed method uses two implicit constraints, namely, quantile regression and advantage weighted regression. In order to assess the efficacy of the proposed algorithm with real controllers, a hardware-in-the-loop test is conducted. The findings demonstrate that by leveraging the generalization capability of the quantile regression to estimate the value of the best available state-action pair and derive the policy using the advantage-weighted form of behavioral cloning to favor actions that receive higher advantage, the proposed data-driven RL can not only learn effectively from a static offline dataset, but also exhibit a robust policy improvement during the subsequent fine-tuning process.
- 17 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Transportation Electrification
- Vol. 10 (1) , 1257-1273
A Hybrid Algorithm Combining Data-Driven and Simulation-Based Reinforcement Learning Approaches to Energy Management of Hybrid Electric Vehicles
Energy management strategy (EMS) is one of the key technologies that improves the fuel efficiency of hybrid electric vehicles (HEVs) by governing the energy flow between the fuel tank and the electric energy storage. With the rapid development of artificial intelligence especially after the great success of AlphaGo, reinforcement learning (RL) has opened up a new window for EMS. Although many RL-based solutions have been successfully applied to EMS tasks, most current RL-based approaches only consider RL as an offline optimization tool, i.e., RL is used to solve optimization problems given a simulation model. This method may be utilized to realize optimal control in simulation, but there is no guarantee that a well-behaved performance can also be achieved in the real-physical world due to the simulation-to-real gap. Meanwhile, for many industrial EMS tasks especially when optimizing an existing controller, a high-fidelity simulator may be unavailable, but some logging data generated by some suboptimal existing controller and simple models of the principal powertrain components neglecting detailed component dynamics could exist. In this context, a hybrid algorithm combining data-driven and simulation-based RL approach is proposed that tries to learn experience from both the real logging data and simulated simple model. Based on the hardware-in-the-loop (HIL) results, a near optimal policy can be obtained (fuel consumption is decreased by approximately 6.10%) by this hybrid algorithm from a small batch logging data and a simple simulated model when compared with the dynamic programming (DP) method using a high-fidelity simulation model mimicking the real-physical world.
- 17 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Transportation Electrification
- Vol. 10 (1) , 1257-1273
A Hybrid Algorithm Combining Data-Driven and Simulation-Based Reinforcement Learning Approaches to Energy Management of Hybrid Electric Vehicles
Energy management strategy (EMS) is one of the key technologies that improves the fuel efficiency of hybrid electric vehicles (HEVs) by governing the energy flow between the fuel tank and the electric energy storage. With the rapid development of artificial intelligence especially after the great success of AlphaGo, reinforcement learning (RL) has opened up a new window for EMS. Although many RL-based solutions have been successfully applied to EMS tasks, most current RL-based approaches only consider RL as an offline optimization tool, i.e., RL is used to solve optimization problems given a simulation model. This method may be utilized to realize optimal control in simulation, but there is no guarantee that a well-behaved performance can also be achieved in the real-physical world due to the simulation-to-real gap. Meanwhile, for many industrial EMS tasks especially when optimizing an existing controller, a high-fidelity simulator may be unavailable, but some logging data generated by some suboptimal existing controller and simple models of the principal powertrain components neglecting detailed component dynamics could exist. In this context, a hybrid algorithm combining data-driven and simulation-based RL approach is proposed that tries to learn experience from both the real logging data and simulated simple model. Based on the hardware-in-the-loop (HIL) results, a near optimal policy can be obtained (fuel consumption is decreased by approximately 6.10%) by this hybrid algorithm from a small batch logging data and a simple simulated model when compared with the dynamic programming (DP) method using a high-fidelity simulation model mimicking the real-physical world.
- Open Access
- 1 April 2022
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 2219 (1) , 012025
Exploiting Reservoir Computing for Optical Secure Communication
A reservoir computing (RC) based on vertical cavity surface emitting laser (VCSEL) subjected to optical feedback is adopted to realize chaos synchronization in optical chaotic communication. Combined with the application of chaos masking (CMS) technology in chaotic secure communication, chaotic signal generated by distribution feedback (DFB) laser after optical feedback and optical injection, that is used as chaotic carrier of hiding original information, and act as target signal of the RC. Then masked information is used for RC training and transmission. Next, utilizing the weights and error offsets trained out by RC at transmitter, carrier can be reconstructed by RC at receiver. Finally, message is acquired by subtracting the reconstructed carrier from received masked signal. Taking image data transmission in 10 km fiber as an example, the chaos synchronization between the carrier signal and the constructed carrier signal is 0.9935, Q-factor is 10 and bit error rate (BER) is 5×10-28 with simulation.
- 1 April 2022
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 2219 (1) , 012025
Open AccessExploiting Reservoir Computing for Optical Secure Communication
A reservoir computing (RC) based on vertical cavity surface emitting laser (VCSEL) subjected to optical feedback is adopted to realize chaos synchronization in optical chaotic communication. Combined with the application of chaos masking (CMS) technology in chaotic secure communication, chaotic signal generated by distribution feedback (DFB) laser after optical feedback and optical injection, that is used as chaotic carrier of hiding original information, and act as target signal of the RC. Then masked information is used for RC training and transmission. Next, utilizing the weights and error offsets trained out by RC at transmitter, carrier can be reconstructed by RC at receiver. Finally, message is acquired by subtracting the reconstructed carrier from received masked signal. Taking image data transmission in 10 km fiber as an example, the chaos synchronization between the carrier signal and the constructed carrier signal is 0.9935, Q-factor is 10 and bit error rate (BER) is 5×10-28 with simulation.