Pengju Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Pengju Zhang has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Yihong Wu
Chinese Academy of Sciences
5 shared publications - Chao Zhou
Chinese Academy of Sciences
4 shared publications - Junfeng Fan
Chinese Academy of Sciences
4 shared publications - Zeng-Guang Hou
University of Chinese Academy of Sciences
4 shared publications - Fulin Tang
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
3 shared publications - Yaming Ou
Chinese Academy of Sciences
3 shared publications
Latest Publications
- 11 September 2026
- Published by Springer Nature
- p. 116-133
Minute4D: Training High-Fidelity 4D Gaussian Splatting in One Minute
- 11 September 2026
- Published by Springer Nature
- p. 116-133
Minute4D: Training High-Fidelity 4D Gaussian Splatting in One Minute
- 16 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. PP (10518215) , 1
Accurate Absolute Scale with Pseudo Depth Constraint for 3D Sparse Construction of Monocular RGB Image
3D Sparse reconstruction and camera pose estimation from monocular sequences are inherently plagued by scale ambiguity, which prevents the recovery of the scene’s absolute scale. Existing solutions are primarily limited in two aspects: (1) traditional geometric methods often depend on additional sensors or restrictive pre-calibration, lacking general applicability; (2) learning-based approaches frequently suffer from scale biases when faced with domain shifts. To overcome these challenges, this paper introduces a unified framework that deeply integrates monocular depth estimation into a traditional reconstruction pipeline. Our method leverages the robustness of modern depth prediction networks to eliminate cumbersome prior assumptions while employing iterative geometric refinement to ensure precise scale accuracy. The proposed framework begins by estimating depth maps via RGB images. First, an absolute scale factor is initialized from the depth priors to calibrate the initial two-view reconstruction, effectively aligning the model into a metric space. Second, during incremental reconstruction, new points and poses are registered via depth-constrained triangulation, with an iterative weighting strategy to enhance stability. Third, we propose a joint bundle adjustment that harmonizes reprojection and geometric errors using an adaptive Mahalanobis distance, which implicitly optimizes the global scale and refines the scene geometry concurrently. Comprehensive evaluations on the ETH3D, TUM-RGBD, and ICL-NUIM benchmarks demonstrate that our framework achieves state-of-the-art performance in absolute scale recovery. The results confirm that our method successfully maintains high geometric fidelity across diverse scenarios, proving its effectiveness and robustness.
- 16 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. PP (10518215) , 1
Accurate Absolute Scale with Pseudo Depth Constraint for 3D Sparse Construction of Monocular RGB Image
3D Sparse reconstruction and camera pose estimation from monocular sequences are inherently plagued by scale ambiguity, which prevents the recovery of the scene’s absolute scale. Existing solutions are primarily limited in two aspects: (1) traditional geometric methods often depend on additional sensors or restrictive pre-calibration, lacking general applicability; (2) learning-based approaches frequently suffer from scale biases when faced with domain shifts. To overcome these challenges, this paper introduces a unified framework that deeply integrates monocular depth estimation into a traditional reconstruction pipeline. Our method leverages the robustness of modern depth prediction networks to eliminate cumbersome prior assumptions while employing iterative geometric refinement to ensure precise scale accuracy. The proposed framework begins by estimating depth maps via RGB images. First, an absolute scale factor is initialized from the depth priors to calibrate the initial two-view reconstruction, effectively aligning the model into a metric space. Second, during incremental reconstruction, new points and poses are registered via depth-constrained triangulation, with an iterative weighting strategy to enhance stability. Third, we propose a joint bundle adjustment that harmonizes reprojection and geometric errors using an adaptive Mahalanobis distance, which implicitly optimizes the global scale and refines the scene geometry concurrently. Comprehensive evaluations on the ETH3D, TUM-RGBD, and ICL-NUIM benchmarks demonstrate that our framework achieves state-of-the-art performance in absolute scale recovery. The results confirm that our method successfully maintains high geometric fidelity across diverse scenarios, proving its effectiveness and robustness.
- 16 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 1-21
Underwater Pipeline and Cable External Inspection Techniques: A Comprehensive Review
Underwater pipelines play an important role in offshore oil and gas transmission, and underwater cables are vital for global interconnection communication. However, due to the harsh underwater environment, underwater pipelines and cables are easily damaged. In order to complete the inspection of underwater pipelines and cables, researchers have proposed many inspection platforms, detection and tracking methods to improve inspection efficiency and automation level. In this paper, the state-of-the-art techniques for underwater pipeline and cable external inspection are comprehensively reviewed. Firstly, underwater pipeline and cable inspection platforms are classified and introduced, and their advantages and disadvantages are discussed. Then, underwater pipeline and cable detection methods including acoustic method, passive optical method, active optical method and electromagnetic method are introduced, and their performance comparison are provided. Besides, underwater pipeline and cable tracking methods are analyzed. Finally, we discuss the key issues and future directions of underwater pipeline and cable inspection. This review will help researchers better understand the current research progress in underwater pipeline and cable inspection platforms, detection and tracking methods, which provides guidance for further research and deployment.
- 16 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 1-21
Underwater Pipeline and Cable External Inspection Techniques: A Comprehensive Review
Underwater pipelines play an important role in offshore oil and gas transmission, and underwater cables are vital for global interconnection communication. However, due to the harsh underwater environment, underwater pipelines and cables are easily damaged. In order to complete the inspection of underwater pipelines and cables, researchers have proposed many inspection platforms, detection and tracking methods to improve inspection efficiency and automation level. In this paper, the state-of-the-art techniques for underwater pipeline and cable external inspection are comprehensively reviewed. Firstly, underwater pipeline and cable inspection platforms are classified and introduced, and their advantages and disadvantages are discussed. Then, underwater pipeline and cable detection methods including acoustic method, passive optical method, active optical method and electromagnetic method are introduced, and their performance comparison are provided. Besides, underwater pipeline and cable tracking methods are analyzed. Finally, we discuss the key issues and future directions of underwater pipeline and cable inspection. This review will help researchers better understand the current research progress in underwater pipeline and cable inspection platforms, detection and tracking methods, which provides guidance for further research and deployment.
- 14 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (3) , 2594-2601
AING-SLAM: Accurate Implicit Neural Geometry-Aware SLAM With Appearance and Semantics via History-Guided Optimization
In range-based SLAM systems, localization accuracy depends on the quality of geometric maps. Sparse LiDAR scans and noisy depth from RGB-D sensors often yield incomplete or inaccurate reconstructions that degrade pose estimation. Appearance and semantic cues, readily available from onboard RGB and pretrained models, can serve as complementary signals to strengthen geometry. Nevertheless, variations in appearance due to illumination or texture and inconsistencies in semantic labels across frames can hinder geometric optimization if directly used as supervision. To address these challenges, we propose AING-SLAM, an Accurate Implicit Neural Geometry-aware SLAM framework that allows appearance and semantics to effectively strengthen geometry in both mapping and odometry. A unified neural point representation with a lightweight cross-modal decoder integrates geometry, appearance and semantics, enabling auxiliary cues to refine geometry even in sparse or ambiguous regions. For pose tracking, appearance-semantic-aided odometry jointly minimizes SDF, appearance, and semantic residuals with adaptive weighting, improving scan-to-map alignment and reducing drift. To safeguard stability, a history-guided gradient fusion strategy aligns instantaneous updates with long-term optimization trends, mitigating occasional inconsistencies between appearance/semantic cues and SDF-based supervision, thereby strengthening geometric optimization. Extensive experiments on indoor RGB-D and outdoor LiDAR benchmarks demonstrate real-time performance, state-of-the-art localization accuracy, and high-fidelity reconstruction across diverse environments.
- 14 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (3) , 2594-2601
AING-SLAM: Accurate Implicit Neural Geometry-Aware SLAM With Appearance and Semantics via History-Guided Optimization
In range-based SLAM systems, localization accuracy depends on the quality of geometric maps. Sparse LiDAR scans and noisy depth from RGB-D sensors often yield incomplete or inaccurate reconstructions that degrade pose estimation. Appearance and semantic cues, readily available from onboard RGB and pretrained models, can serve as complementary signals to strengthen geometry. Nevertheless, variations in appearance due to illumination or texture and inconsistencies in semantic labels across frames can hinder geometric optimization if directly used as supervision. To address these challenges, we propose AING-SLAM, an Accurate Implicit Neural Geometry-aware SLAM framework that allows appearance and semantics to effectively strengthen geometry in both mapping and odometry. A unified neural point representation with a lightweight cross-modal decoder integrates geometry, appearance and semantics, enabling auxiliary cues to refine geometry even in sparse or ambiguous regions. For pose tracking, appearance-semantic-aided odometry jointly minimizes SDF, appearance, and semantic residuals with adaptive weighting, improving scan-to-map alignment and reducing drift. To safeguard stability, a history-guided gradient fusion strategy aligns instantaneous updates with long-term optimization trends, mitigating occasional inconsistencies between appearance/semantic cues and SDF-based supervision, thereby strengthening geometric optimization. Extensive experiments on indoor RGB-D and outdoor LiDAR benchmarks demonstrate real-time performance, state-of-the-art localization accuracy, and high-fidelity reconstruction across diverse environments.
- 29 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 72 (5) , 5126-5135
Underwater Robot Self-Localization Method Using Tightly Coupled Events, Images, Inertial, and Acoustic Fusion
At present, underwater robot self-localization methods mainly rely on inertial measurement unit (IMU) and Doppler velocity log (DVL) based dead reckoning (DR) and standard camera-based vision localization methods. However, the DR method has the problem of error accumulation with the increase of distance and the standard camera-based vision method is difficult to achieve accurate localization due to little information and sparse features of standard images in low-light underwater environment. To address these problems, a tightly coupled event, standard image, inertia, and acoustic fusion method is proposed to realize robust and accurate self-localization of underwater robots. First, the high-frequency IMU data is fused to generate motion compensation event frames, and the feature points of event frames and standard frames are extracted and tracked respectively. Then, a joint optimization problem of cost function is established by using the reprojection error term of the event frame and standard frame, the IMU inertia error term and the DVL velocity error term, avoiding the explicit switching between event frame and standard frame. Finally, a multisensor-based self-localization system for underwater robots is developed, and a series of event motion compensation and self-localization experiments in real underwater environment are carried out. Experimental results show that the proposed method has better localization accuracy than standard camera, IMU, and DVL fusion methods.
- 29 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 72 (5) , 5126-5135
Underwater Robot Self-Localization Method Using Tightly Coupled Events, Images, Inertial, and Acoustic Fusion
At present, underwater robot self-localization methods mainly rely on inertial measurement unit (IMU) and Doppler velocity log (DVL) based dead reckoning (DR) and standard camera-based vision localization methods. However, the DR method has the problem of error accumulation with the increase of distance and the standard camera-based vision method is difficult to achieve accurate localization due to little information and sparse features of standard images in low-light underwater environment. To address these problems, a tightly coupled event, standard image, inertia, and acoustic fusion method is proposed to realize robust and accurate self-localization of underwater robots. First, the high-frequency IMU data is fused to generate motion compensation event frames, and the feature points of event frames and standard frames are extracted and tracked respectively. Then, a joint optimization problem of cost function is established by using the reprojection error term of the event frame and standard frame, the IMU inertia error term and the DVL velocity error term, avoiding the explicit switching between event frame and standard frame. Finally, a multisensor-based self-localization system for underwater robots is developed, and a series of event motion compensation and self-localization experiments in real underwater environment are carried out. Experimental results show that the proposed method has better localization accuracy than standard camera, IMU, and DVL fusion methods.
- 22 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 72 (3) , 2821-2831
Hybrid-VINS: Underwater Tightly Coupled Hybrid Visual Inertial Dense SLAM for AUV
Traditional visual simultaneous localization and mapping (SLAM) methods primarily rely on passive vision, such as monocular cameras, which often exhibit reduced accuracy in localization and struggle to create dense maps in low-light underwater conditions. In this article, a tightly coupled hybrid visual inertial navigation system (VINS), named Hybrid-VINS, is proposed by fuzing active and passive vision, which enables robust localization and dense mapping underwater. Specifically, a self-designed active vision device called underwater binocular structured light (UBSL) is integrated to provide more accurate depth estimation of the scene, enhancing the initialization and visual feature tracking process of monocular VINS. As well as it addresses the deficiency of VINS in dense mapping. In addition, a more robust hybrid vision-aided loop closure detection algorithm is proposed to mitigate issues stemming from pure passive vision misjudgments and mismatches. Furthermore, an underwater autonomous hybrid vision system is developed in both simulated and real-world underwater environments to collect various datasets for verifying the performance of Hybrid-VINS. Experimental results demonstrate that, compared with passive VINS, Hybrid-VINS holds greater promise for underwater high-precision localization and dense mapping.
- 22 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 72 (3) , 2821-2831
Hybrid-VINS: Underwater Tightly Coupled Hybrid Visual Inertial Dense SLAM for AUV
Traditional visual simultaneous localization and mapping (SLAM) methods primarily rely on passive vision, such as monocular cameras, which often exhibit reduced accuracy in localization and struggle to create dense maps in low-light underwater conditions. In this article, a tightly coupled hybrid visual inertial navigation system (VINS), named Hybrid-VINS, is proposed by fuzing active and passive vision, which enables robust localization and dense mapping underwater. Specifically, a self-designed active vision device called underwater binocular structured light (UBSL) is integrated to provide more accurate depth estimation of the scene, enhancing the initialization and visual feature tracking process of monocular VINS. As well as it addresses the deficiency of VINS in dense mapping. In addition, a more robust hybrid vision-aided loop closure detection algorithm is proposed to mitigate issues stemming from pure passive vision misjudgments and mismatches. Furthermore, an underwater autonomous hybrid vision system is developed in both simulated and real-world underwater environments to collect various datasets for verifying the performance of Hybrid-VINS. Experimental results demonstrate that, compared with passive VINS, Hybrid-VINS holds greater promise for underwater high-precision localization and dense mapping.
- 16 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (6) , 5158-5165
RSS: Robust Stereo SLAM With Novel Extraction and Full Exploitation of Plane Features
Planar structures, prevalent in man-made environments, can be observed by a camera for significant periods of time due to their large spatial presence. These structures provide strong planar regularities for Simultaneous Localization and Mapping (SLAM) systems, facilitating long-term navigation. Therefore, we propose a novel point-plane-based stereo SLAM system, fully regularized by plane features within a unified non-linear optimization framework. The core of our method is an accurate and efficient stereo plane extraction algorithm with strict 2D and 3D outlier rejection mechanisms, effectively extracting main planes from robust stereo correspondences and enabling real-time point-plane association. Furthermore, we introduce a novel optimization formulation, incorporating geometric feature (point and plane) and across-feature (point-on-plane) constraints that promote each other through the mutual constraints between associated point and plane features, which fully exploits plane constraints to improve the performance of SLAM system. The proposed plane extraction algorithm is evaluated on the EuRoC MAV dataset, achieving significant improvements in number, accuracy, reliability, and efficiency over the state-of-the-art (SOTA) stereo point-plane-based system (Zhang et al., 2021). The results of an ablation study on two public datasets show that the proposed SLAM system outperforms (Zhang et al., 2021) in both accuracy and robustness, and further demonstrate the mutual enhancement between the two types of constraints.
- 16 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (6) , 5158-5165
RSS: Robust Stereo SLAM With Novel Extraction and Full Exploitation of Plane Features
Planar structures, prevalent in man-made environments, can be observed by a camera for significant periods of time due to their large spatial presence. These structures provide strong planar regularities for Simultaneous Localization and Mapping (SLAM) systems, facilitating long-term navigation. Therefore, we propose a novel point-plane-based stereo SLAM system, fully regularized by plane features within a unified non-linear optimization framework. The core of our method is an accurate and efficient stereo plane extraction algorithm with strict 2D and 3D outlier rejection mechanisms, effectively extracting main planes from robust stereo correspondences and enabling real-time point-plane association. Furthermore, we introduce a novel optimization formulation, incorporating geometric feature (point and plane) and across-feature (point-on-plane) constraints that promote each other through the mutual constraints between associated point and plane features, which fully exploits plane constraints to improve the performance of SLAM system. The proposed plane extraction algorithm is evaluated on the EuRoC MAV dataset, achieving significant improvements in number, accuracy, reliability, and efficiency over the state-of-the-art (SOTA) stereo point-plane-based system (Zhang et al., 2021). The results of an ablation study on two public datasets show that the proposed SLAM system outperforms (Zhang et al., 2021) in both accuracy and robustness, and further demonstrate the mutual enhancement between the two types of constraints.
- 15 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 5383-5397
Covariant Peak Constraint for Accurate Keypoint Detection and Keypoint-Specific Descriptor Learning
Local feature extraction consists of keypoint detection and local descriptor extraction. Firstly, in keypoint detector learning, existing covariance constraint loss functions cannot constrain the probability distribution shapes in local probability maps that surround keypoints. And existing auxiliary peak loss functions, which are used to alleviate the problem, impair the performance of local feature methods. To solve this problem, we propose a novel Covariant Peak constraint Loss (CP Loss) which is defined as the expectations of local probability maps' position errors. Minimizing our CP Loss can make local probability maps accurately peak at reliable keypoints. Secondly, in descriptor learning, the Neural Reprojection Error (NRE) aims at constraining dense descriptor maps of images. But we argue that only those descriptors of keypoints need to be constrained. Thus, we propose a novel Conditional Neural Reprojection Error (CNRE) that is only conditioned on keypoints. Compared with the NRE, our CNRE can achieve much higher efficiency and produce more keypoint-specific descriptors with better matching performance. We use our CP Loss and CNRE to train a local feature network named as CPCN-Feat. Experimental results show that our CPCN-Feat achieves state-of-the-art performance on four challenging benchmarks.
- 15 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 5383-5397
Covariant Peak Constraint for Accurate Keypoint Detection and Keypoint-Specific Descriptor Learning
Local feature extraction consists of keypoint detection and local descriptor extraction. Firstly, in keypoint detector learning, existing covariance constraint loss functions cannot constrain the probability distribution shapes in local probability maps that surround keypoints. And existing auxiliary peak loss functions, which are used to alleviate the problem, impair the performance of local feature methods. To solve this problem, we propose a novel Covariant Peak constraint Loss (CP Loss) which is defined as the expectations of local probability maps' position errors. Minimizing our CP Loss can make local probability maps accurately peak at reliable keypoints. Secondly, in descriptor learning, the Neural Reprojection Error (NRE) aims at constraining dense descriptor maps of images. But we argue that only those descriptors of keypoints need to be constrained. Thus, we propose a novel Conditional Neural Reprojection Error (CNRE) that is only conditioned on keypoints. Compared with the NRE, our CNRE can achieve much higher efficiency and produce more keypoint-specific descriptors with better matching performance. We use our CP Loss and CNRE to train a local feature network named as CPCN-Feat. Experimental results show that our CPCN-Feat achieves state-of-the-art performance on four challenging benchmarks.
- 7 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 29 (3) , 1997-2008
Structured Light Vision 3-D Reconstruction System for Different Media Considering Refraction: Design, Modeling, and Calibration
Due to the advantages of high measurement accuracy and good robustness, structured light vision (SLV) methods have been extensively applied for 3-D reconstruction in air and water scene. However, most current SLV systems cannot adapt to different media without recalibration, and have the disadvantages of complicated calibration process including SLV model parameter calibration in air and water. In order to overcome these problems, a SLV 3-D reconstruction system for different media considering refraction is proposed. First, an accurate SLV measurement model is established for 3-D reconstruction in different media, which considers structured light plane refraction model and camera refraction model simultaneously. Specifically, the relationship of the SLV model parameters in air and water are proposed. Second, a simple and efficient calibration method of SLV model parameters is presented. After SLV model calibration in air and refraction interface parameters calibration, the SLV model parameters in different media can be obtained accurately without recalibration. Finally, a compact SLV system is developed, and calibration experiments and 3-D reconstruction experiments in air and different turbid water are conducted. Experiment results show that the proposed system could obtain satisfactory 3-D reconstruction performance in air and different turbidity water.
- 7 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 29 (3) , 1997-2008
Structured Light Vision 3-D Reconstruction System for Different Media Considering Refraction: Design, Modeling, and Calibration
Due to the advantages of high measurement accuracy and good robustness, structured light vision (SLV) methods have been extensively applied for 3-D reconstruction in air and water scene. However, most current SLV systems cannot adapt to different media without recalibration, and have the disadvantages of complicated calibration process including SLV model parameter calibration in air and water. In order to overcome these problems, a SLV 3-D reconstruction system for different media considering refraction is proposed. First, an accurate SLV measurement model is established for 3-D reconstruction in different media, which considers structured light plane refraction model and camera refraction model simultaneously. Specifically, the relationship of the SLV model parameters in air and water are proposed. Second, a simple and efficient calibration method of SLV model parameters is presented. After SLV model calibration in air and refraction interface parameters calibration, the SLV model parameters in different media can be obtained accurately without recalibration. Finally, a compact SLV system is developed, and calibration experiments and 3-D reconstruction experiments in air and different turbid water are conducted. Experiment results show that the proposed system could obtain satisfactory 3-D reconstruction performance in air and different turbidity water.