Chuhao Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Chuhao Liu has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Shaojie Shen
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
8 shared publications - Ke Wang
School of Information Engineering, Changan University, Xian, China
5 shared publications - Zhijian Qiao
Department of Electronic and Computer Engineering, the Hong Kong University of Science and Technology, Hong Kong, China
3 shared publications - Jieqi Shi
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
2 shared publications - Boyu Zhou
Southern University of Science and Technology
1 shared publication - Fangwei Xiao
College of Information Engineering, Changan University, Xian, China
1 shared publication
Latest Publications
- 5 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 41 (15523098) , 3870-3889
SG-Reg: Generalizable and Efficient Scene Graph Registration
This paper addresses the challenges of registering two rigid semantic scene graphs, an essential capability when an autonomous agent needs to register its map against a remote agent, or against a prior map. The hand-crafted descriptors in classical semantic-aided registration, or the ground-truth annotation reliance in learning-based scene graph registration, impede their application in practical real-world environments. To address the challenges, we design a scene graph network to encode multiple modalities of semantic nodes: open-set semantic feature, local topology with spatial awareness, and shape feature. These modalities are fused to create compact semantic node features. The matching layers then search for correspondences in a coarse-to-fine manner. In the back-end, we employ a robust pose estimator to decide transformation according to the correspondences. We manage to maintain a sparse and hierarchical scene representation. Our approach demands fewer GPU resources and fewer communication bandwidth in multi-agent tasks. Moreover, we design a new data generation approach using vision foundation models and a semantic mapping module to reconstruct semantic scene graphs. It differs significantly from previous works, which rely on ground-truth semantic annotations to generate data. We validate our method in a two-agent SLAM benchmark. It significantly outperforms the hand-crafted baseline in terms of registration success rate. Compared to visual loop closure networks, our method achieves a slightly higher registration recall while requiring only 52 KB of communication bandwidth for each query frame. Code available at: http://github.com/HKUST-Aerial-Robotics/SG-Reg
- 5 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 41 (15523098) , 3870-3889
SG-Reg: Generalizable and Efficient Scene Graph Registration
This paper addresses the challenges of registering two rigid semantic scene graphs, an essential capability when an autonomous agent needs to register its map against a remote agent, or against a prior map. The hand-crafted descriptors in classical semantic-aided registration, or the ground-truth annotation reliance in learning-based scene graph registration, impede their application in practical real-world environments. To address the challenges, we design a scene graph network to encode multiple modalities of semantic nodes: open-set semantic feature, local topology with spatial awareness, and shape feature. These modalities are fused to create compact semantic node features. The matching layers then search for correspondences in a coarse-to-fine manner. In the back-end, we employ a robust pose estimator to decide transformation according to the correspondences. We manage to maintain a sparse and hierarchical scene representation. Our approach demands fewer GPU resources and fewer communication bandwidth in multi-agent tasks. Moreover, we design a new data generation approach using vision foundation models and a semantic mapping module to reconstruct semantic scene graphs. It differs significantly from previous works, which rely on ground-truth semantic annotations to generate data. We validate our method in a two-agent SLAM benchmark. It significantly outperforms the hand-crafted baseline in terms of registration success rate. Compared to visual loop closure networks, our method achieves a slightly higher registration recall while requiring only 52 KB of communication bandwidth for each query frame. Code available at: http://github.com/HKUST-Aerial-Robotics/SG-Reg
- 7 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 22282-22295
Speak the Same Language: Global LiDAR Registration on BIM Using Pose Hough Transform
Light detection and ranging (LiDAR) point clouds and building information modeling (BIM) represent two distinct data modalities in the fields of robot perception and construction. These modalities originate from different sources and are associated with unique reference frames. The primary goal of this study is to align these modalities within a shared reference frame using a global registration approach, effectively enabling them to “speak the same language”. To achieve this, we propose a cross-modality registration method, spanning from the front end to the back end. At the front end, we extract triangle descriptors by identifying walls and intersected corners, enabling the matching of corner triplets with a complexity independent of the BIM’s size. For the back-end transformation estimation, we utilize the Hough transform to map the matched triplets to the transformation space and introduce a hierarchical voting mechanism to hypothesize multiple pose candidates. The final transformation is then verified using our designed occupancy-aware scoring method. To assess the effectiveness of our approach, we conducted real-world multi-session experiments in a large-scale university building, employing two different types of LiDAR sensors. We make the collected datasets and codes publicly available to benefit the community.
- 7 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 22282-22295
Speak the Same Language: Global LiDAR Registration on BIM Using Pose Hough Transform
Light detection and ranging (LiDAR) point clouds and building information modeling (BIM) represent two distinct data modalities in the fields of robot perception and construction. These modalities originate from different sources and are associated with unique reference frames. The primary goal of this study is to align these modalities within a shared reference frame using a global registration approach, effectively enabling them to “speak the same language”. To achieve this, we propose a cross-modality registration method, spanning from the front end to the back end. At the front end, we extract triangle descriptors by identifying walls and intersected corners, enabling the matching of corner triplets with a complexity independent of the BIM’s size. For the back-end transformation estimation, we utilize the Hough transform to map the matched triplets to the transformation space and introduce a hierarchical voting mechanism to hypothesize multiple pose candidates. The final transformation is then verified using our designed occupancy-aware scoring method. To assess the effectiveness of our approach, we conducted real-world multi-session experiments in a large-scale university building, employing two different types of LiDAR sensors. We make the collected datasets and codes publicly available to benefit the community.
- 24 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (7) , 6400-6407
Multi-View Depth Estimation by Using Adaptive Point Graph to Fuse Single-View Depth Probabilities
Recently, some methods estimate depth maps by fusing several adjacent single-view depth probabilities. They have achieved promising performance in multi-view inconsistent areas, such as texture-less surfaces, reflective surfaces, and moving objects. However, these methods involve two new problems: their thin cost volumes contain many invalid values, and the depths of adjacent volume units tend to be very different, which hinders the valid fusion of multi-view information. To deal with these issues, we design a novel point graph based single-views fusing method to estimate depth maps from several sequential images. Our method first estimates the initial probabilistic distribution of the depth map for input images, the distribution is parameterized as a pixel-wise depth and uncertainty. Then, we sample non-uniform depth candidates from the reference image's initial distribution. Diverse from the popular 3D cost volume, we utilize sampled depth candidates to construct an adaptive local point graph to represent multi-view geometric constraints. For pixels with multi-view consistency, we aggregate their local graphs to update their initial depths. And take the updated pixels as control points to refine the depth of the remaining pixels. We demonstrate the effectiveness of the proposed method by quantitative and qualitative comparisons with recent baseline works on the KITTI Odometry dataset and the DADD dataset, and our results surpass all competing methods even without 3D cost volume.
- 24 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (7) , 6400-6407
Multi-View Depth Estimation by Using Adaptive Point Graph to Fuse Single-View Depth Probabilities
Recently, some methods estimate depth maps by fusing several adjacent single-view depth probabilities. They have achieved promising performance in multi-view inconsistent areas, such as texture-less surfaces, reflective surfaces, and moving objects. However, these methods involve two new problems: their thin cost volumes contain many invalid values, and the depths of adjacent volume units tend to be very different, which hinders the valid fusion of multi-view information. To deal with these issues, we design a novel point graph based single-views fusing method to estimate depth maps from several sequential images. Our method first estimates the initial probabilistic distribution of the depth map for input images, the distribution is parameterized as a pixel-wise depth and uncertainty. Then, we sample non-uniform depth candidates from the reference image's initial distribution. Diverse from the popular 3D cost volume, we utilize sampled depth candidates to construct an adaptive local point graph to represent multi-view geometric constraints. For pixels with multi-view consistency, we aggregate their local graphs to update their initial depths. And take the updated pixels as control points to refine the depth of the remaining pixels. We demonstrate the effectiveness of the proposed method by quantitative and qualitative comparisons with recent baseline works on the KITTI Odometry dataset and the DADD dataset, and our results surpass all competing methods even without 3D cost volume.
- 18 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (3) , 2232-2239
FM-Fusion: Instance-Aware Semantic Mapping Boosted by Vision-Language Foundation Models
Semantic mapping based on the supervised object detectors is sensitive to image distribution. In real-world environments, the object detection and segmentation performance can lead to a major drop, preventing the use of semantic mapping in a wider domain. On the other hand, the development of vision-language foundation models demonstrates a strong zero-shot transferability across data distribution. It provides an opportunity to construct generalizable instance-aware semantic maps. Hence, this work explores how to boost instance-aware semantic mapping from object detection generated from foundation models. We propose a probabilistic label fusion method to predict close-set semantic classes from open-set label measurements. An instance refinement module merges the over-segmented instances caused by inconsistent segmentation. We integrate all the modules into a unified semantic mapping system. Reading a sequence of RGB-D input, our work incrementally reconstructs an instance-aware semantic map. We evaluate the zero-shot performance of our method in ScanNet and SceneNN datasets. Our method achieves 40.3 mean average precision (mAP) on the ScanNet semantic instance segmentation task. It outperforms the traditional semantic mapping method significantly.
- 18 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (3) , 2232-2239
FM-Fusion: Instance-Aware Semantic Mapping Boosted by Vision-Language Foundation Models
Semantic mapping based on the supervised object detectors is sensitive to image distribution. In real-world environments, the object detection and segmentation performance can lead to a major drop, preventing the use of semantic mapping in a wider domain. On the other hand, the development of vision-language foundation models demonstrates a strong zero-shot transferability across data distribution. It provides an opportunity to construct generalizable instance-aware semantic maps. Hence, this work explores how to boost instance-aware semantic mapping from object detection generated from foundation models. We propose a probabilistic label fusion method to predict close-set semantic classes from open-set label measurements. An instance refinement module merges the over-segmented instances caused by inconsistent segmentation. We integrate all the modules into a unified semantic mapping system. Reading a sequence of RGB-D input, our work incrementally reconstructs an instance-aware semantic map. We evaluate the zero-shot performance of our method in ScanNet and SceneNN datasets. Our method achieves 40.3 mean average precision (mAP) on the ScanNet semantic instance segmentation task. It outperforms the traditional semantic mapping method significantly.
- Open Access
Towards View-invariant and Accurate Loop Detection Based on Scene Graph
Loop detection plays a key role in visual Simultaneous Localization and Mapping (SLAM) by correcting the accumulated pose drift. In indoor scenarios, the richly distributed semantic landmarks are view-point invariant and hold strong descriptive power in loop detection. The current semantic-aided loop detection embeds the topology between semantic instances to search a loop. However, current semantic-aided loop detection methods face challenges in dealing with ambiguous semantic instances and drastic viewpoint differences, which are not fully addressed in the literature. This paper introduces a novel loop detection method based on an incrementally created scene graph, targeting the visual SLAM at indoor scenes. It jointly considers the macro-view topology, micro-view topology, and occupancy of semantic instances to find correct correspondences. Experiments using handheld RGB-D sequence show our method is able to accurately detect loops in drastically changed viewpoints. It maintains a high precision in observing objects with similar topology and appearance. Our method also demonstrates that it is robust in changed indoor scenes.
Open AccessTowards View-invariant and Accurate Loop Detection Based on Scene Graph
Loop detection plays a key role in visual Simultaneous Localization and Mapping (SLAM) by correcting the accumulated pose drift. In indoor scenarios, the richly distributed semantic landmarks are view-point invariant and hold strong descriptive power in loop detection. The current semantic-aided loop detection embeds the topology between semantic instances to search a loop. However, current semantic-aided loop detection methods face challenges in dealing with ambiguous semantic instances and drastic viewpoint differences, which are not fully addressed in the literature. This paper introduces a novel loop detection method based on an incrementally created scene graph, targeting the visual SLAM at indoor scenes. It jointly considers the macro-view topology, micro-view topology, and occupancy of semantic instances to find correct correspondences. Experiments using handheld RGB-D sequence show our method is able to accurately detect loops in drastically changed viewpoints. It maintains a high precision in observing objects with similar topology and appearance. Our method also demonstrates that it is robust in changed indoor scenes.
- Open Access
Geometric Calibration for Cameras with Inconsistent Imaging Capabilities
Traditional calibration methods rely on the accurate localization of the chessboard points in images and their maximum likelihood estimation (MLE)-based optimization models implicitly require all detected points to have an identical uncertainty. The uncertainties of the detected control points are mainly determined by camera pose, the slant of the chessboard and the inconsistent imaging capabilities of the camera. The negative influence of the uncertainties that are induced by the two former factors can be eliminated by adequate data sampling. However, the last factor leads to the detected control points from some sensor areas having larger uncertainties than those from other sensor areas. This causes the final calibrated parameters to overfit the control points that are located at the poorer sensor areas. In this paper, we present a method for measuring the uncertainties of the detected control points and incorporating these measured uncertainties into the optimization model of the geometric calibration. The new model suppresses the influence from the control points with large uncertainties while amplifying the contributions from points with small uncertainties for the final convergence. We demonstrate the usability of the proposed method by first using eight cameras to collect a calibration dataset and then comparing our method to other recent works and the calibration module in OpenCV using that dataset.
Open AccessGeometric Calibration for Cameras with Inconsistent Imaging Capabilities
Traditional calibration methods rely on the accurate localization of the chessboard points in images and their maximum likelihood estimation (MLE)-based optimization models implicitly require all detected points to have an identical uncertainty. The uncertainties of the detected control points are mainly determined by camera pose, the slant of the chessboard and the inconsistent imaging capabilities of the camera. The negative influence of the uncertainties that are induced by the two former factors can be eliminated by adequate data sampling. However, the last factor leads to the detected control points from some sensor areas having larger uncertainties than those from other sensor areas. This causes the final calibrated parameters to overfit the control points that are located at the poorer sensor areas. In this paper, we present a method for measuring the uncertainties of the detected control points and incorporating these measured uncertainties into the optimization model of the geometric calibration. The new model suppresses the influence from the control points with large uncertainties while amplifying the contributions from points with small uncertainties for the final convergence. We demonstrate the usability of the proposed method by first using eight cameras to collect a calibration dataset and then comparing our method to other recent works and the calibration module in OpenCV using that dataset.
- 3 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 3160-3167
Depth Estimation Under Motion With Single Pair Rolling Shutter Stereo Images
Many methods have been proposed to process stereo matching for rolling shutter image pairs, they treat all pixels from an image pair in an identical way and require additional estimation approaches to estimate motion states of the camera. However, pixels from a rolling shutter image pair naturally have diverse baseline lengths, and motion estimation methods are unstable for one instantaneous image pair input. In this letter, we present a rolling shutter stereo depth estimation pipeline, which can robustly estimate motion states and depth maps by alternating the estimation of the depth maps and refining the motion states from coarse image levels to fine image levels. What is more, we design a novel cost volume building method for rolling shutter image pairs, which adapts depth candidates to the change of baseline lengths for all pixels. We further demonstrate the usability of the proposed method by constructing a new platform, building an outdoor evaluation dataset, and comparing it with baseline methods.
- 3 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 3160-3167
Depth Estimation Under Motion With Single Pair Rolling Shutter Stereo Images
Many methods have been proposed to process stereo matching for rolling shutter image pairs, they treat all pixels from an image pair in an identical way and require additional estimation approaches to estimate motion states of the camera. However, pixels from a rolling shutter image pair naturally have diverse baseline lengths, and motion estimation methods are unstable for one instantaneous image pair input. In this letter, we present a rolling shutter stereo depth estimation pipeline, which can robustly estimate motion states and depth maps by alternating the estimation of the depth maps and refining the motion states from coarse image levels to fine image levels. What is more, we design a novel cost volume building method for rolling shutter image pairs, which adapts depth candidates to the change of baseline lengths for all pixels. We further demonstrate the usability of the proposed method by constructing a new platform, building an outdoor evaluation dataset, and comparing it with baseline methods.
- Top Cited Papers
- 10 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 4 (4) , 3529-3536
Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flight
In this letter, we propose a robust and efficient quadrotor motion planning system for fast flight in three-dimensional complex environments. We adopt a kinodynamic path searching method to find a safe, kinodynamic feasible, and minimum-time initial trajectory in the discretized control space. We improve the smoothness and clearance of the trajectory by a B-spline optimization, which incorporates gradient information from a Euclidean distance field and dynamic constraints efficiently utilizing the convex hull property of B-spline. Finally, by representing the final trajectory as a non-uniform B-spline, an iterative time adjustment method is adopted to guarantee dynamically feasible and non-conservative trajectories. We validate our proposed method in various complex simulational environments. The competence of the method is also validated in challenging real-world tasks. We release our code as an open-source package.
- 10 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 4 (4) , 3529-3536
Top Cited PapersRobust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flight
In this letter, we propose a robust and efficient quadrotor motion planning system for fast flight in three-dimensional complex environments. We adopt a kinodynamic path searching method to find a safe, kinodynamic feasible, and minimum-time initial trajectory in the discretized control space. We improve the smoothness and clearance of the trajectory by a B-spline optimization, which incorporates gradient information from a Euclidean distance field and dynamic constraints efficiently utilizing the convex hull property of B-spline. Finally, by representing the final trajectory as a non-uniform B-spline, an iterative time adjustment method is adopted to guarantee dynamically feasible and non-conservative trajectories. We validate our proposed method in various complex simulational environments. The competence of the method is also validated in challenging real-world tasks. We release our code as an open-source package.