Zhixing Song
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Zhixing Song has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Xuebo Zhang
Institute of Robotics and Automatic Information Systems and Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
9 shared publications - Shiyong Zhang
College of Artificial Intelligence, Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
8 shared publications - Jing Yuan
College of Artificial Intelligence, Nankai University, Tianjin, China
6 shared publications - Runhua Wang
Tianjin Key Laboratory of Intelligent Robotics, the College of Artificial Intelligence, and the Institute of Robotics and Automatic Information System, Nankai University, Tianjin, China
5 shared publications - Youwei Wang
College of Artificial Intelligence, Institute of Robotics and Automatic Information System, and the Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
5 shared publications - Songyang Wu
College of Artificial Intelligence, Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
4 shared publications
Latest Publications
- 5 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 1114-1121
DOGL-SLAM: Dynamic Object-Level SLAM via Joint Gaussian-Landmark Tracking
Recent advancements in 3D Gaussian Splatting (3DGS) have significantly improved the mapping quality and computational efficiency of visual Simultaneous Localization and Mapping (SLAM). We propose DOGL-SLAM, a novel framework that integrates 3DGS into its core pipeline, enabling accurate camera pose tracking, object-level interaction, and high-fidelity scene reconstruction in dynamic environments. Firstly, a joint graph optimization module incorporates both dense Gaussian and sparse landmark constraints, enabling precise alignment between camera tracking and mapping. Secondly, a consistent object-level semantic fusion module embeds category labels into 3D Gaussians, grouping them via semantic consistency and optimizing distributions through cross-view losses to support scene manipulation. Finally, a hierarchical dynamic filtering pipeline is introduced, consisting of segmentation, dynamic feature exclusion, gradient masking, and visibility-aware Gaussian pruning across three parallel threads. Our system is evaluated across multiple datasets, showing a significant improvement in high-quality view synthesis for dynamic scenes. Additionally, the generated object-level semantic maps facilitate advanced downstream tasks, highlighting the potential of a robust SLAM framework. The source code and supplementary video will be released.
- 5 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 1114-1121
DOGL-SLAM: Dynamic Object-Level SLAM via Joint Gaussian-Landmark Tracking
Recent advancements in 3D Gaussian Splatting (3DGS) have significantly improved the mapping quality and computational efficiency of visual Simultaneous Localization and Mapping (SLAM). We propose DOGL-SLAM, a novel framework that integrates 3DGS into its core pipeline, enabling accurate camera pose tracking, object-level interaction, and high-fidelity scene reconstruction in dynamic environments. Firstly, a joint graph optimization module incorporates both dense Gaussian and sparse landmark constraints, enabling precise alignment between camera tracking and mapping. Secondly, a consistent object-level semantic fusion module embeds category labels into 3D Gaussians, grouping them via semantic consistency and optimizing distributions through cross-view losses to support scene manipulation. Finally, a hierarchical dynamic filtering pipeline is introduced, consisting of segmentation, dynamic feature exclusion, gradient masking, and visibility-aware Gaussian pruning across three parallel threads. Our system is evaluated across multiple datasets, showing a significant improvement in high-quality view synthesis for dynamic scenes. Additionally, the generated object-level semantic maps facilitate advanced downstream tasks, highlighting the potential of a robust SLAM framework. The source code and supplementary video will be released.
- 1 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Aerospace and Electronic Systems
- Vol. 61 (6) , 17567-17583
SPTL-LCC: Single-Shot, Pixel-Level, Target-Free, and LiDAR-Type Agnostic LiDAR-Camera Extrinsic Self-Calibration
The fusion of LiDAR and camera data holds vast application prospects, with calibration as a crucial prerequisite. In this paper, we propose SPTL-LCC, a LiDAR -type agnostic and target-free LiDAR-camera extrinsic calibration framework. The core of SPTL-LCC lies in extracting accurate 3D-2D correspondences in 2D image space using a densified point cloud and an image. To ensure the acquisition of a densified point cloud, a point cloud accumulation evaluation metric is proposed. Building upon this, a multi-stage LiDAR image processing and optimization method based on a virtual camera is proposed. Then, multiple image matching networks are employed to estimate 2D-2D correspondences between the LiDAR images and camera images, resulting in 3D-2D correspondences. A progressive reprojection error minimization algorithm is subsequently proposed to optimize the extrinsic parameters based on these correspondences. SPTL-LCC improves LiDAR image quality in a few iterations, enhancing correspondence estimation accuracy, thus achieving coarse-to-fine calibration. Extensive experiments on four self-recorded datasets show SPTL-LCC outperforms state-of-the-art methods in terms of robustness, accuracy, and practicality, with an average error of less than 0.03 m in translation and 0.4$^{\circ }$ in rotation. Moreover, experiments on the KITTI dataset show SPTL-LCC even achieves comparable performance to end-to-end methods trained on this dataset, which exhibit poor generalization. Our code and datasets are open-source: https://github.com/NKU-MobFly-Robotics/SPTG-LCC.
- 1 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Aerospace and Electronic Systems
- Vol. 61 (6) , 17567-17583
SPTL-LCC: Single-Shot, Pixel-Level, Target-Free, and LiDAR-Type Agnostic LiDAR-Camera Extrinsic Self-Calibration
The fusion of LiDAR and camera data holds vast application prospects, with calibration as a crucial prerequisite. In this paper, we propose SPTL-LCC, a LiDAR -type agnostic and target-free LiDAR-camera extrinsic calibration framework. The core of SPTL-LCC lies in extracting accurate 3D-2D correspondences in 2D image space using a densified point cloud and an image. To ensure the acquisition of a densified point cloud, a point cloud accumulation evaluation metric is proposed. Building upon this, a multi-stage LiDAR image processing and optimization method based on a virtual camera is proposed. Then, multiple image matching networks are employed to estimate 2D-2D correspondences between the LiDAR images and camera images, resulting in 3D-2D correspondences. A progressive reprojection error minimization algorithm is subsequently proposed to optimize the extrinsic parameters based on these correspondences. SPTL-LCC improves LiDAR image quality in a few iterations, enhancing correspondence estimation accuracy, thus achieving coarse-to-fine calibration. Extensive experiments on four self-recorded datasets show SPTL-LCC outperforms state-of-the-art methods in terms of robustness, accuracy, and practicality, with an average error of less than 0.03 m in translation and 0.4$^{\circ }$ in rotation. Moreover, experiments on the KITTI dataset show SPTL-LCC even achieves comparable performance to end-to-end methods trained on this dataset, which exhibit poor generalization. Our code and datasets are open-source: https://github.com/NKU-MobFly-Robotics/SPTG-LCC.
- 6 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 74 (00189456) , 1-12
FI-SLAM: Feature Information-Based Robust and Efficient LiDAR SLAM
In environments with sparse or redundant features, the accuracy or real-time performance of LiDAR simultaneous localization and mapping (SLAM) systems significantly deteriorates. This paper proposes a novel feature information-based robust and efficient LiDAR SLAM (FI-SLAM). First, we introduce a single-feature information evaluation method based on subspace vector similarity to cope with the scale inconsistency between rotation and translation in 3-D space. Then, to overcome the limitations of existing degradation evaluation methods that only focus on constraint sufficiency, we elaborately devise a robust degradation evaluation algorithm based on total information volume and density, which considers both the sufficiency and reliability of constraints with strong robustness to environmental changes. Information density is a novel concept we introduce, referring to the average information of feature constraints, used to evaluate constraint reliability. Next, to reduce the delay of the SLAM system, we propose an efficient and reliable feature selection algorithm based on information priority, which can quickly select a feature subset with high-quality information and requires almost no additional computing time. This algorithm significantly improves system efficiency in environments with redundant features. Finally, experimental results in self-built simulation scenarios and real-world environments show that FI-SLAM outperforms state-of-the-art methods in terms of accuracy, robustness, and real-time performance. Specifically, the average time consumption for feature association and optimization is reduced by 48.39%, and the localization accuracy in degenerate environments is improved by up to 89.77%. Supplementary video is available at https://youtu.be/x9duvizvvIg.
- 6 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 74 (00189456) , 1-12
FI-SLAM: Feature Information-Based Robust and Efficient LiDAR SLAM
In environments with sparse or redundant features, the accuracy or real-time performance of LiDAR simultaneous localization and mapping (SLAM) systems significantly deteriorates. This paper proposes a novel feature information-based robust and efficient LiDAR SLAM (FI-SLAM). First, we introduce a single-feature information evaluation method based on subspace vector similarity to cope with the scale inconsistency between rotation and translation in 3-D space. Then, to overcome the limitations of existing degradation evaluation methods that only focus on constraint sufficiency, we elaborately devise a robust degradation evaluation algorithm based on total information volume and density, which considers both the sufficiency and reliability of constraints with strong robustness to environmental changes. Information density is a novel concept we introduce, referring to the average information of feature constraints, used to evaluate constraint reliability. Next, to reduce the delay of the SLAM system, we propose an efficient and reliable feature selection algorithm based on information priority, which can quickly select a feature subset with high-quality information and requires almost no additional computing time. This algorithm significantly improves system efficiency in environments with redundant features. Finally, experimental results in self-built simulation scenarios and real-world environments show that FI-SLAM outperforms state-of-the-art methods in terms of accuracy, robustness, and real-time performance. Specifically, the average time consumption for feature association and optimization is reduced by 48.39%, and the localization accuracy in degenerate environments is improved by up to 89.77%. Supplementary video is available at https://youtu.be/x9duvizvvIg.
- Open Access
VS-SLAM: Robust SLAM Based on LiDAR Loop Closure Detection with Virtual Descriptors and Selective Memory Storage in Challenging Environments
LiDAR loop closure detection is a key technology to mitigate localization drift in LiDAR SLAM, but it remains challenging in structurally similar environments and memory-constrained platforms. This paper proposes VS-SLAM, a novel and robust SLAM system that leverages virtual descriptors and selective memory storage to enhance LiDAR loop closure detection in challenging environments. Firstly, to mitigate the sensitivity of existing descriptors to translational changes, we propose a novel virtual descriptor technique that enhances translational invariance and improves loop closure detection accuracy. Then, to further improve the accuracy of loop closure detection in structurally similar environments, we propose an efficient and reliable selective memory storage technique based on scene recognition and key descriptor evaluation, which also reduces the memory consumption of the loop closure database. Next, based on the two proposed techniques, we develop a LiDAR SLAM system with loop closure detection capability, which maintains high accuracy and robustness even in challenging environments with structural similarity. Finally, extensive experiments in self-built simulation, real-world environments, and public datasets demonstrate that VS-SLAM outperforms state-of-the-art methods in terms of memory efficiency, accuracy, and robustness. Specifically, the memory consumption of the loop closure database is reduced by an average of 92.86% compared with SC-LVI-SAM and VS-SLAM-w/o-st, and the localization accuracy in structurally similar challenging environments is improved by an average of 66.41% compared with LVI-SAM.
Open AccessVS-SLAM: Robust SLAM Based on LiDAR Loop Closure Detection with Virtual Descriptors and Selective Memory Storage in Challenging Environments
LiDAR loop closure detection is a key technology to mitigate localization drift in LiDAR SLAM, but it remains challenging in structurally similar environments and memory-constrained platforms. This paper proposes VS-SLAM, a novel and robust SLAM system that leverages virtual descriptors and selective memory storage to enhance LiDAR loop closure detection in challenging environments. Firstly, to mitigate the sensitivity of existing descriptors to translational changes, we propose a novel virtual descriptor technique that enhances translational invariance and improves loop closure detection accuracy. Then, to further improve the accuracy of loop closure detection in structurally similar environments, we propose an efficient and reliable selective memory storage technique based on scene recognition and key descriptor evaluation, which also reduces the memory consumption of the loop closure database. Next, based on the two proposed techniques, we develop a LiDAR SLAM system with loop closure detection capability, which maintains high accuracy and robustness even in challenging environments with structural similarity. Finally, extensive experiments in self-built simulation, real-world environments, and public datasets demonstrate that VS-SLAM outperforms state-of-the-art methods in terms of memory efficiency, accuracy, and robustness. Specifically, the memory consumption of the loop closure database is reduced by an average of 92.86% compared with SC-LVI-SAM and VS-SLAM-w/o-st, and the localization accuracy in structurally similar challenging environments is improved by an average of 66.41% compared with LVI-SAM.
- 10 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (9) , 7421-7428
TC$^{2}$LI-SLAM: A Tightly-Coupled Camera-LiDAR-Inertial SLAM System
In this letter, we propose TC $^{2}$ LI-SLAM, a novel and tightly-coupled camera-LiDAR-inertial SLAM system with a separated front end and a unified back end. The front end accepts non-identical frequency inputs and performs real-time pose estimation while incorporating different inputs into keyframes for the back end, benefiting from our novel keyframe-oriented data synchronization method. To improve the accuracy of the back-end optimization, we innovatively leverage multi-sensor consecutive co-vision constraints in a keyframe-based local bundle adjustment problem. To bridge the gap between different residual terms from different sensors in the optimization problem, we propose a cross-modal residual standardization method that can be easily applied. Furthermore, the proposed system can degrade into TC $^{2}$ L-SLAM, a tightly-coupled camera-LiDAR SLAM system without IMU, to cope with situations where inertial measurements are not available or reliable. Finally, both TC $^{2}$ LI-SLAM and TC $^{2}$ L-SLAM are tested on public and self-recorded datasets. The results show their superior performance in terms of accuracy compared with the state-of-the-art SLAM systems, with at most 33% accuracy improvement over ORB-SLAM3 on the KITTI dataset.
- 10 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (9) , 7421-7428
TC$^{2}$LI-SLAM: A Tightly-Coupled Camera-LiDAR-Inertial SLAM System
In this letter, we propose TC $^{2}$ LI-SLAM, a novel and tightly-coupled camera-LiDAR-inertial SLAM system with a separated front end and a unified back end. The front end accepts non-identical frequency inputs and performs real-time pose estimation while incorporating different inputs into keyframes for the back end, benefiting from our novel keyframe-oriented data synchronization method. To improve the accuracy of the back-end optimization, we innovatively leverage multi-sensor consecutive co-vision constraints in a keyframe-based local bundle adjustment problem. To bridge the gap between different residual terms from different sensors in the optimization problem, we propose a cross-modal residual standardization method that can be easily applied. Furthermore, the proposed system can degrade into TC $^{2}$ L-SLAM, a tightly-coupled camera-LiDAR SLAM system without IMU, to cope with situations where inertial measurements are not available or reliable. Finally, both TC $^{2}$ LI-SLAM and TC $^{2}$ L-SLAM are tested on public and self-recorded datasets. The results show their superior performance in terms of accuracy compared with the state-of-the-art SLAM systems, with at most 33% accuracy improvement over ORB-SLAM3 on the KITTI dataset.
- 25 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (2) , 1061-1071
MPOC-SLAM: An RGB-D SLAM System With Motion Probability and Object Category in High Dynamic Environments
The performance of simultaneous localization and mapping (SLAM) is significantly impacted by dynamic objects. Existing dynamic SLAM methods still suffer from some major issues, such as insufficient use of semantic information, losing track when dynamic objects occupy the field of view, and inability to achieve real-time processing. In this article, we present SLAM with motion probability and object category (MPOC-SLAM), an RGB-D SLAM system capable of high-performance, real-time work in high dynamic environments. Considering intensive semantic information, our method introduces the concepts of motion probability (MP) and object category (OC), and tightly integrates them within the whole SLAM system. First, a priority feature extraction strategy is proposed based on MP, and then a dual-constraint feature matching method is designed with the adaptive probability threshold and category consistency. Both methods reduce the influence of dynamic objects in the extraction and association process. Second, to speed up the process of calculating reprojection errors, a semantic-guided geometric method with the partitioning hypothetico-deductive method is presented. The semantic-guided geometric method and instance segmentation are then fused through the variance weighted fusion with their confidences. Finally, as a more important part of SLAM, dynamic-aware weights are added to the original pose optimization, which assigns larger weights to static feature points and map points to reduce their alteration during optimization. We perform evaluations on the TUM public dataset and real-world environments. Compared with ORB-SLAM2, the percentage reduction of root mean square error of absolute trajectory error is as high as 98.72%. Results show that MPOC-SLAM realizes a significant improvement in accuracy as well as a more robust and real-time performance compared with other state-of-the-art dynamic SLAM methods.
- 25 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (2) , 1061-1071
MPOC-SLAM: An RGB-D SLAM System With Motion Probability and Object Category in High Dynamic Environments
The performance of simultaneous localization and mapping (SLAM) is significantly impacted by dynamic objects. Existing dynamic SLAM methods still suffer from some major issues, such as insufficient use of semantic information, losing track when dynamic objects occupy the field of view, and inability to achieve real-time processing. In this article, we present SLAM with motion probability and object category (MPOC-SLAM), an RGB-D SLAM system capable of high-performance, real-time work in high dynamic environments. Considering intensive semantic information, our method introduces the concepts of motion probability (MP) and object category (OC), and tightly integrates them within the whole SLAM system. First, a priority feature extraction strategy is proposed based on MP, and then a dual-constraint feature matching method is designed with the adaptive probability threshold and category consistency. Both methods reduce the influence of dynamic objects in the extraction and association process. Second, to speed up the process of calculating reprojection errors, a semantic-guided geometric method with the partitioning hypothetico-deductive method is presented. The semantic-guided geometric method and instance segmentation are then fused through the variance weighted fusion with their confidences. Finally, as a more important part of SLAM, dynamic-aware weights are added to the original pose optimization, which assigns larger weights to static feature points and map points to reduce their alteration during optimization. We perform evaluations on the TUM public dataset and real-world environments. Compared with ORB-SLAM2, the percentage reduction of root mean square error of absolute trajectory error is as high as 98.72%. Results show that MPOC-SLAM realizes a significant improvement in accuracy as well as a more robust and real-time performance compared with other state-of-the-art dynamic SLAM methods.
- 4 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 10 (1) , 117-129
Target-Free and User-Friendly Online Extrinsic Calibration of LiDAR-IMU-Camera Systems Guided by Motion Excitation Assessment
Robust and reliable calibration forms the foundation of efficient multi-sensor fusion. Most existing calibration methods are offline and rely on artificial targets, which is time consuming and unfriendly to non-expert users. To improve efficiency, robustness and user-friendliness, this paper proposes a novel target-free LiDAR-IMU-camera online extrinsic calibration framework. An IMU-centric motion excitation assessment method is newly designed to assist non-expert users in data collection, serving as the termination criterion for extrinsic rotation initialization. We calibrate the extrinsic parameters of the LiDAR and camera relative to the IMU in real-time and in parallel without any priori knowledge: (i) For camera-IMU calibration, a new convergence criterion based on the coefficient of variation is proposed to judge if the calibration has been accomplished. (ii) For LiDAR-IMU calibration, a data selection strategy is designed to concurrently record a segment of original data from both the LiDAR and IMU during initialization. Subsequently, the selected data is utilized to refine the extrinsic parameters under a continuous-time batch optimization framework. Moreover, a scene recognition algorithm is proposed to autonomously select parameters according to indoor or outdoor features. Finally, we leverage the IMU as a bridge for computing the LiDAR-camera extrinsic parameters. Extensive real experiments on our self-recorded and public datasets show the high accuracy and robustness of the proposed method, with an average error of less than 0.01m in translation and $0.5^{\circ }$ in rotation. (Supplementary video link: https://youtu.be/Iph0kd_LIlk )
- 4 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 10 (1) , 117-129
Target-Free and User-Friendly Online Extrinsic Calibration of LiDAR-IMU-Camera Systems Guided by Motion Excitation Assessment
Robust and reliable calibration forms the foundation of efficient multi-sensor fusion. Most existing calibration methods are offline and rely on artificial targets, which is time consuming and unfriendly to non-expert users. To improve efficiency, robustness and user-friendliness, this paper proposes a novel target-free LiDAR-IMU-camera online extrinsic calibration framework. An IMU-centric motion excitation assessment method is newly designed to assist non-expert users in data collection, serving as the termination criterion for extrinsic rotation initialization. We calibrate the extrinsic parameters of the LiDAR and camera relative to the IMU in real-time and in parallel without any priori knowledge: (i) For camera-IMU calibration, a new convergence criterion based on the coefficient of variation is proposed to judge if the calibration has been accomplished. (ii) For LiDAR-IMU calibration, a data selection strategy is designed to concurrently record a segment of original data from both the LiDAR and IMU during initialization. Subsequently, the selected data is utilized to refine the extrinsic parameters under a continuous-time batch optimization framework. Moreover, a scene recognition algorithm is proposed to autonomously select parameters according to indoor or outdoor features. Finally, we leverage the IMU as a bridge for computing the LiDAR-camera extrinsic parameters. Extensive real experiments on our self-recorded and public datasets show the high accuracy and robustness of the proposed method, with an average error of less than 0.01m in translation and $0.5^{\circ }$ in rotation. (Supplementary video link: https://youtu.be/Iph0kd_LIlk )
- 5 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 28 (4) , 1920-1929
IR-VIO: Illumination-Robust Visual-Inertial Odometry Based on Adaptive Weighting Algorithm With Two-Layer Confidence Maximization
Illumination change, image blur, and fast motion dramatically decrease the performance of visual-inertial navigation systems (VINS). This article presents a new illumination-robust visual-inertial odometry (IR-VIO) based on adaptive weighting algorithm with two-layer confidence maximization. First, to prevent the VIO performance degradation caused by poor image quality in complex scenes and ignoring the confidence differences of feature points, we develop a novel adaptive weighting algorithm on the multisensor layer and visual feature layer to better fuse multisensor information and maximize the overall confidence of VIO. Second, to solve the problems of image feature tracking difficulty and excessive image noise in illumination-changing scenes, an image enhancement algorithm is introduced to enhance consecutive images to the same brightness level, while a block noise removal algorithm with constraint protection mechanism is proposed to dynamically remove noise points. Finally, experimental results in the public dataset and real-world environments demonstrate that IR-VIO has superior performance in terms of accuracy and robustness compared with the state-of-the-art methods.
- 5 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 28 (4) , 1920-1929
IR-VIO: Illumination-Robust Visual-Inertial Odometry Based on Adaptive Weighting Algorithm With Two-Layer Confidence Maximization
Illumination change, image blur, and fast motion dramatically decrease the performance of visual-inertial navigation systems (VINS). This article presents a new illumination-robust visual-inertial odometry (IR-VIO) based on adaptive weighting algorithm with two-layer confidence maximization. First, to prevent the VIO performance degradation caused by poor image quality in complex scenes and ignoring the confidence differences of feature points, we develop a novel adaptive weighting algorithm on the multisensor layer and visual feature layer to better fuse multisensor information and maximize the overall confidence of VIO. Second, to solve the problems of image feature tracking difficulty and excessive image noise in illumination-changing scenes, an image enhancement algorithm is introduced to enhance consecutive images to the same brightness level, while a block noise removal algorithm with constraint protection mechanism is proposed to dynamically remove noise points. Finally, experimental results in the public dataset and real-world environments demonstrate that IR-VIO has superior performance in terms of accuracy and robustness compared with the state-of-the-art methods.
- Open Access
- 16 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
IR-VIO: Illumination-Robust Visual-Inertial Odometry Based on Adaptive Weighting Algorithm With Two-Layer Confidence Maximization
Illumination change, image blur, and fast motion dramatically decrease the performance of visual-inertial navigation systems (VINS). This paper presents a new illumination-robust visual-inertial odometry (IR-VIO) based on adaptive weighting algorithm with two-layer confidence maximization. First, to prevent the VIO performance degradation caused by poor image quality in complex scenes and ignoring the confidence differences of feature points, we develop a novel adaptive weighting algorithm on the multi-sensor layer and visual feature layer to better fuse multi-sensor information and maximize the overall confidence of VIO. Second, to solve the problems of image feature tracking difficulty and excessive image noise in illumination-changing scenes, an image enhancement algorithm is introduced to enhance consecutive images to the same brightness level, while a block noise removal algorithm with constraint protection mechanism is proposed to dynamically remove feature points in noise areas. Finally, experimental results on the public dataset and real-world environments demonstrate that IR-VIO has superior performance in terms of accuracy and robustness compared with the state-of-the-art methods. Supplementary video is available at https://youtu.be/163Sk7twEMY.
- 16 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessIR-VIO: Illumination-Robust Visual-Inertial Odometry Based on Adaptive Weighting Algorithm With Two-Layer Confidence Maximization
Illumination change, image blur, and fast motion dramatically decrease the performance of visual-inertial navigation systems (VINS). This paper presents a new illumination-robust visual-inertial odometry (IR-VIO) based on adaptive weighting algorithm with two-layer confidence maximization. First, to prevent the VIO performance degradation caused by poor image quality in complex scenes and ignoring the confidence differences of feature points, we develop a novel adaptive weighting algorithm on the multi-sensor layer and visual feature layer to better fuse multi-sensor information and maximize the overall confidence of VIO. Second, to solve the problems of image feature tracking difficulty and excessive image noise in illumination-changing scenes, an image enhancement algorithm is introduced to enhance consecutive images to the same brightness level, while a block noise removal algorithm with constraint protection mechanism is proposed to dynamically remove feature points in noise areas. Finally, experimental results on the public dataset and real-world environments demonstrate that IR-VIO has superior performance in terms of accuracy and robustness compared with the state-of-the-art methods. Supplementary video is available at https://youtu.be/163Sk7twEMY.