Sheng Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Sheng Zhang has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Xiangtao Zhuan
School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China
5 shared publications - Joseph Mathew
Indian Institute of Science
1 shared publication Jun Cheng
CAS Key Laboratory of Human‐Machine Intelligence‐Synergy Systems Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen People's Republic of China
1 shared publication- Jun Cheng
Chinese Academy of Sciences
1 shared publication Lian Huai
AIoT CTO, BOE Technology Group Company, Ltd., Beijing, China
1 shared publication- Lin Ma
Meituan, Beijing, China
1 shared publication
Latest Publications
- 13 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (10) , 10019-10026
CMG3D: Compensation Towards Modality Gap for Open-Vocabulary Indoor 3D Object Detection
For open-vocabulary indoor three-dimensional (3D) object detection (OVI3DOD), there is a gap between the image and the point cloud for indoor scenes, especially on distant objects. However, existing algorithms ignore this problem, which weakens the detection performance. Therefore, we propose Compensation towards the Modal Gap for open-vocabulary indoor 3D object detection (CMG3D). CMG3D consists of three modules: multimodal compensation (MC), object proposal filtering (OPF) and pseudo label refinement and generation (PLRG). In the MC, features from images are converted into the pseudo voxel space and then summed with the voxel space of the point cloud, which is used to compensate for the modality gap, while the OPF filters the object proposals to avoid confusion between the foreground and background. Finally, in the PLRG, the predictions from the two-dimensional (2D) detector are refined by the multimodal large language model (LLM) SigLIP and then transformed into 3D pseudo labels for the training process. Finally, we evaluate CMG3D on two indoor datasets, SUN RGB-D and ScanNet, and achieve state-of-the-art results.
- 13 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (10) , 10019-10026
CMG3D: Compensation Towards Modality Gap for Open-Vocabulary Indoor 3D Object Detection
For open-vocabulary indoor three-dimensional (3D) object detection (OVI3DOD), there is a gap between the image and the point cloud for indoor scenes, especially on distant objects. However, existing algorithms ignore this problem, which weakens the detection performance. Therefore, we propose Compensation towards the Modal Gap for open-vocabulary indoor 3D object detection (CMG3D). CMG3D consists of three modules: multimodal compensation (MC), object proposal filtering (OPF) and pseudo label refinement and generation (PLRG). In the MC, features from images are converted into the pseudo voxel space and then summed with the voxel space of the point cloud, which is used to compensate for the modality gap, while the OPF filters the object proposals to avoid confusion between the foreground and background. Finally, in the PLRG, the predictions from the two-dimensional (2D) detector are refined by the multimodal large language model (LLM) SigLIP and then transformed into 3D pseudo labels for the training process. Finally, we evaluate CMG3D on two indoor datasets, SUN RGB-D and ScanNet, and achieve state-of-the-art results.
- 28 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (10) , 6523-6530
TBFNT3D: Two-Branch Fusion Network With Transformer for Multimodal Indoor 3D Object Detection
Indoor 3D object detection based on point clouds has been widely applied for robotics, augmented reality and virtual reality. The point clouds generated from RGB-D cameras are sparse for distant objects, which affects the detection performance. Multimodal 3D object detection can improve the detection performance by fusing features for point clouds and images. RGB images can be converted to dense 3D features, which can be applied as a complement to 3D object detection using only point clouds. We refer to the 3D data transformed from RGB images as estimated 3D data. Therefore, we propose a two-branch fusion network with a transformer for multimodal indoor 3D object detection named TBFNT3D. In TBFNT3D, voxels converted from the point clouds and images are added together to obtain a consistent voxel representation. The features for the voxel space are enriched, and features from different modalities do not require a complex alignment process. To make better use of estimated 3D data, we need to process noise and remove redundant estimated 3D data. The receptive field for 3D sparse convolution is expanded into the 2D image space, which weakens the effect of noise. A bin-based sampling strategy is applied for near objects and distant objects, removing the redundant estimated 3D data. In addition, to fuse the multimodal features efficiently, we apply a deformable transformer to obtain the detection results. Finally, TBFNT3D is evaluated on the SUN RGB-D dataset and ScanNet dataset, and state-of-the-art results are achieved.
- 28 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (10) , 6523-6530
TBFNT3D: Two-Branch Fusion Network With Transformer for Multimodal Indoor 3D Object Detection
Indoor 3D object detection based on point clouds has been widely applied for robotics, augmented reality and virtual reality. The point clouds generated from RGB-D cameras are sparse for distant objects, which affects the detection performance. Multimodal 3D object detection can improve the detection performance by fusing features for point clouds and images. RGB images can be converted to dense 3D features, which can be applied as a complement to 3D object detection using only point clouds. We refer to the 3D data transformed from RGB images as estimated 3D data. Therefore, we propose a two-branch fusion network with a transformer for multimodal indoor 3D object detection named TBFNT3D. In TBFNT3D, voxels converted from the point clouds and images are added together to obtain a consistent voxel representation. The features for the voxel space are enriched, and features from different modalities do not require a complex alignment process. To make better use of estimated 3D data, we need to process noise and remove redundant estimated 3D data. The receptive field for 3D sparse convolution is expanded into the 2D image space, which weakens the effect of noise. A bin-based sampling strategy is applied for near objects and distant objects, removing the redundant estimated 3D data. In addition, to fuse the multimodal features efficiently, we apply a deformable transformer to obtain the detection results. Finally, TBFNT3D is evaluated on the SUN RGB-D dataset and ScanNet dataset, and state-of-the-art results are achieved.
- Open Access
Distributed Model Predictive Control for Two-Dimensional Electric Vehicle Platoon Based on QMIX Algorithm
In this paper, an improved distributed model predictive control (IDMPC) method for the platoon consisting of electric vehicles is put forward. And the motion of the platoon is performed in two dimensions, which contains longitudinal motion and lateral motion. Firstly, a platoon model is built based on the car-following model for a single following vehicle. Then, the IDMPC strategy is designed with the consideration of multiple objectives. The symmetrical weight matrices in the IDMPC are important for the final control effect. To control each following vehicle in the platoon coordinately, the weights for the IDMPC are optimized based on the QMIX algorithm in multi-agent reinforcement learning. The QMIX can fully consider the global information in the multi-vehicle following process; therefore, the IDMPC can get optimal control variables. Finally, the simulation and experimental results verify the IDMPC. Compared to the comparison strategies, the IDMPC has the better lane tracking, stability in lateral direction and economic performance.
Open AccessDistributed Model Predictive Control for Two-Dimensional Electric Vehicle Platoon Based on QMIX Algorithm
In this paper, an improved distributed model predictive control (IDMPC) method for the platoon consisting of electric vehicles is put forward. And the motion of the platoon is performed in two dimensions, which contains longitudinal motion and lateral motion. Firstly, a platoon model is built based on the car-following model for a single following vehicle. Then, the IDMPC strategy is designed with the consideration of multiple objectives. The symmetrical weight matrices in the IDMPC are important for the final control effect. To control each following vehicle in the platoon coordinately, the weights for the IDMPC are optimized based on the QMIX algorithm in multi-agent reinforcement learning. The QMIX can fully consider the global information in the multi-vehicle following process; therefore, the IDMPC can get optimal control variables. Finally, the simulation and experimental results verify the IDMPC. Compared to the comparison strategies, the IDMPC has the better lane tracking, stability in lateral direction and economic performance.
- Open Access
Two-Dimensional Car-Following Control Strategy for Electric Vehicle Based on MPC and DQN
For the coupling problem of longitudinal control and lateral control of vehicles, a two-dimensional (2-D) car-following control strategy for an electric vehicle is proposed in this paper. First, a 2-D car-following model for longitudinal following and lateral lane keeping is established. Then, a 2-D car-following control strategy is designed, and the longitudinal following control and lateral lane keeping control are integrated into one model predictive control (MPC) framework. The 2-D car-following strategy can realize the multi-objective coordinated optimization for longitudinal control and lateral control during the 2-D car-following process, and the multiple objectives are: safety, tracking, comfort, lane keeping, lateral stability and economy. In addition, the economy is important for electric vehicles. The weight matrix of the objective function in the MPC framework is symmetric, and the weight coefficients for the weight matrix have a great influence on the control. The contribution of this paper is: in order to adapt to different dynamic processes of lane keeping, the weight coefficients in the MPC framework are optimized in real-time based on the deep Q network (DQN) algorithm. Finally, to verify the 2-D car-following control strategy, a comparison strategy and two experimental scenarios are set, and simulation experiments are carried out. In scenario 1, compared with the comparison strategy, the lane keeping, lateral stability and economy of the proposed strategy are improved by 37.21%, 17.57% and 9.26%, respectively. In scenario 2, compared with the comparison strategy, the lane keeping, lateral stability and economy of the proposed strategy are improved by 36.45%, 16.66% and 18.52%, respectively. Therefore, compared with the comparison strategy, the 2-D car-following control strategy can have better lane keeping, lateral stability and economy on the premise of ensuring other performances during the 2-D car-following process.
Open AccessTwo-Dimensional Car-Following Control Strategy for Electric Vehicle Based on MPC and DQN
For the coupling problem of longitudinal control and lateral control of vehicles, a two-dimensional (2-D) car-following control strategy for an electric vehicle is proposed in this paper. First, a 2-D car-following model for longitudinal following and lateral lane keeping is established. Then, a 2-D car-following control strategy is designed, and the longitudinal following control and lateral lane keeping control are integrated into one model predictive control (MPC) framework. The 2-D car-following strategy can realize the multi-objective coordinated optimization for longitudinal control and lateral control during the 2-D car-following process, and the multiple objectives are: safety, tracking, comfort, lane keeping, lateral stability and economy. In addition, the economy is important for electric vehicles. The weight matrix of the objective function in the MPC framework is symmetric, and the weight coefficients for the weight matrix have a great influence on the control. The contribution of this paper is: in order to adapt to different dynamic processes of lane keeping, the weight coefficients in the MPC framework are optimized in real-time based on the deep Q network (DQN) algorithm. Finally, to verify the 2-D car-following control strategy, a comparison strategy and two experimental scenarios are set, and simulation experiments are carried out. In scenario 1, compared with the comparison strategy, the lane keeping, lateral stability and economy of the proposed strategy are improved by 37.21%, 17.57% and 9.26%, respectively. In scenario 2, compared with the comparison strategy, the lane keeping, lateral stability and economy of the proposed strategy are improved by 36.45%, 16.66% and 18.52%, respectively. Therefore, compared with the comparison strategy, the 2-D car-following control strategy can have better lane keeping, lateral stability and economy on the premise of ensuring other performances during the 2-D car-following process.
- Open Access
- 11 July 2022
- Published by Institution of Engineering and Technology (IET) in Electronics Letters
- Vol. 58 (18) , 696-698
BMFN3D: Bidirectional multilayer fusion network for indoor 3D object detection
Among various indoor 3D object detection methods, one of which is based on 3D convolution. However, in current 3D convolutional method for indoor 3D object detection, feature fusion is only implemented in a single direction and adjacent layer, which leads to the feature loss during feature propagation and weakens the detection performance. To solve the above problem, a bidirectional multilayer fusion network (BMFN) is proposed to enrich features for prediction. Furthermore, we propose a new network based on BMFN for 3D object detection, BMFN3D. In BMFN3D, the channel and spatial attention mechanisms are introduced into the backbone and head of detection network, so as to enhance the selection of effective feature information. Finally, a new calculation method for regression loss is proposed with the consideration of the 3D spatial information. We validate our method with two indoor datasets of SUN RGB‐D and ScanNet, and achieve state‐of‐the‐art results.
- 11 July 2022
- Published by Institution of Engineering and Technology (IET) in Electronics Letters
- Vol. 58 (18) , 696-698
Open AccessBMFN3D: Bidirectional multilayer fusion network for indoor 3D object detection
Among various indoor 3D object detection methods, one of which is based on 3D convolution. However, in current 3D convolutional method for indoor 3D object detection, feature fusion is only implemented in a single direction and adjacent layer, which leads to the feature loss during feature propagation and weakens the detection performance. To solve the above problem, a bidirectional multilayer fusion network (BMFN) is proposed to enrich features for prediction. Furthermore, we propose a new network based on BMFN for 3D object detection, BMFN3D. In BMFN3D, the channel and spatial attention mechanisms are introduced into the backbone and head of detection network, so as to enhance the selection of effective feature information. Finally, a new calculation method for regression loss is proposed with the consideration of the 3D spatial information. We validate our method with two indoor datasets of SUN RGB‐D and ScanNet, and achieve state‐of‐the‐art results.
- Open Access
Study on Adaptive Cruise Control Strategy for Battery Electric Vehicle Considering Weight Adjustment
This paper studies control strategies for adaptive cruise control (ACC) systems in battery electric vehicles (BEVs). A hierarchical control structure is adopted for the ACC system, and the structure contains an upper controller and a lower controller. This paper focuses on the upper controller. In the upper controller, model predictive control (MPC) is applied for optimizing multiple objectives in the car-following process. In addition, multiple objectives, including safety, tracking, comfort, and energy economy, can be transformed into a symmetric objective function with constraints in MPC. In the objective function, the corresponding weight matrix for the optimization of multiple objectives is implemented in symmetric form to reduce the computational complexity. The weights in the weight matrix are usually set to be constant. However, the motion states of the own vehicle and the front vehicle change with respect to time during a car-following process, resulting in variation of the driving conditions. MPCs with constant weights do not adapt well to changes in driving conditions, which limits the performance of the ACC system. Therefore, a strategy for weight adjustment is proposed in order to improve the tracking performance, in which some weights in MPC can be adjusted according to the relative velocity of two vehicles in real time. The simulation experiments are carried out to demonstrate the effectiveness of the strategy for weight adjustment. Based on achieving the other control objectives, the ACC system with the weight adjustment has better tracking performance than the ACC system with the constant weight. While the tracking is improved, the energy economy is also improved.
Open AccessStudy on Adaptive Cruise Control Strategy for Battery Electric Vehicle Considering Weight Adjustment
This paper studies control strategies for adaptive cruise control (ACC) systems in battery electric vehicles (BEVs). A hierarchical control structure is adopted for the ACC system, and the structure contains an upper controller and a lower controller. This paper focuses on the upper controller. In the upper controller, model predictive control (MPC) is applied for optimizing multiple objectives in the car-following process. In addition, multiple objectives, including safety, tracking, comfort, and energy economy, can be transformed into a symmetric objective function with constraints in MPC. In the objective function, the corresponding weight matrix for the optimization of multiple objectives is implemented in symmetric form to reduce the computational complexity. The weights in the weight matrix are usually set to be constant. However, the motion states of the own vehicle and the front vehicle change with respect to time during a car-following process, resulting in variation of the driving conditions. MPCs with constant weights do not adapt well to changes in driving conditions, which limits the performance of the ACC system. Therefore, a strategy for weight adjustment is proposed in order to improve the tracking performance, in which some weights in MPC can be adjusted according to the relative velocity of two vehicles in real time. The simulation experiments are carried out to demonstrate the effectiveness of the strategy for weight adjustment. Based on achieving the other control objectives, the ACC system with the weight adjustment has better tracking performance than the ACC system with the constant weight. While the tracking is improved, the energy economy is also improved.
- Open Access
- 1 January 2019
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2019 (1)
Study on Adaptive Cruise Control Strategy for Battery Electric Vehicle
This paper studies the control strategy for adaptive cruise control (ACC) system on a battery electric vehicle (BEV) in the car‐following process, and the highlight of this paper is that the regeneration braking of BEV is considered in the car‐following process. The hierarchical control structure is adopted for the ACC system. And the structure contains an upper controller and a lower controller. In the upper controller, multiple objectives including the safety, tracking, comfort, and energy consumption are optimized by using the model predictive control (MPC) method. In the lower controller, the energy is recovered during braking. So the energy economy is improved by reducing energy consumption and increasing energy recovery. The proposed ACC strategy is evaluated in simulation experiment. In the simulation experiment, safe tracking for the front vehicle is guaranteed, and the comfort and the energy economy are improved greatly. So the proposed adaptive cruise control strategy can make ACC more widely used in BEVs.
- 1 January 2019
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2019 (1)
Open AccessStudy on Adaptive Cruise Control Strategy for Battery Electric Vehicle
This paper studies the control strategy for adaptive cruise control (ACC) system on a battery electric vehicle (BEV) in the car‐following process, and the highlight of this paper is that the regeneration braking of BEV is considered in the car‐following process. The hierarchical control structure is adopted for the ACC system. And the structure contains an upper controller and a lower controller. In the upper controller, multiple objectives including the safety, tracking, comfort, and energy consumption are optimized by using the model predictive control (MPC) method. In the lower controller, the energy is recovered during braking. So the energy economy is improved by reducing energy consumption and increasing energy recovery. The proposed ACC strategy is evaluated in simulation experiment. In the simulation experiment, safe tracking for the front vehicle is guaranteed, and the comfort and the energy economy are improved greatly. So the proposed adaptive cruise control strategy can make ACC more widely used in BEVs.
- Open Access
- 1 January 2019
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2019 (1)
Model‐Predictive Optimization for Pure Electric Vehicle during a Vehicle‐Following Process
In this paper, a pure electric vehicle (PEV) equipped with adaptive cruise control (ACC) system is studied for a vehicle‐following process. And a multiobjective optimization algorithm for ACC system is proposed in a model‐predictive control (MPC) framework for optimizing safety, tracking capability, driving comfortability and energy consumption. The longitudinal dynamics of the ACC system are modeled, which not only considers the vehicle spacing and speed, but also introduces the acceleration and the change rate of acceleration (jerk) for the host vehicle and fully considers the influence of the acceleration of the leading vehicle. The improvement of driving comfortability and the reduction of energy consumption are achieved mainly by optimizing the acceleration and jerk of host vehicle. Some optimized reference trajectories are introduced to MPC for improving driving comfortability of host vehicle. The performances of the multiobjective upper level algorithm combined with the PEV model are evaluated for three representative scenarios. The results demonstrate the effectiveness of the proposed algorithm.
- 1 January 2019
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2019 (1)
Open AccessModel‐Predictive Optimization for Pure Electric Vehicle during a Vehicle‐Following Process
In this paper, a pure electric vehicle (PEV) equipped with adaptive cruise control (ACC) system is studied for a vehicle‐following process. And a multiobjective optimization algorithm for ACC system is proposed in a model‐predictive control (MPC) framework for optimizing safety, tracking capability, driving comfortability and energy consumption. The longitudinal dynamics of the ACC system are modeled, which not only considers the vehicle spacing and speed, but also introduces the acceleration and the change rate of acceleration (jerk) for the host vehicle and fully considers the influence of the acceleration of the leading vehicle. The improvement of driving comfortability and the reduction of energy consumption are achieved mainly by optimizing the acceleration and jerk of host vehicle. Some optimized reference trajectories are introduced to MPC for improving driving comfortability of host vehicle. The performances of the multiobjective upper level algorithm combined with the PEV model are evaluated for three representative scenarios. The results demonstrate the effectiveness of the proposed algorithm.
- 1 January 2006
- Published by Springer Nature
- p. 352-359
Feature Group Optimization for Machinery Fault Diagnosis Based on Fuzzy Measures
- 1 January 2006
- Published by Springer Nature
- p. 352-359
Feature Group Optimization for Machinery Fault Diagnosis Based on Fuzzy Measures