Xuecheng Xu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Xuecheng Xu has a total of 50 co-authors. Here are the top co-authors by number of shared publications:
- Rong Xiong
Zhejiang University
14 shared publications - Yue Wang
Zhejiang University
11 shared publications - Sha Lu
Zhejiang University, Hangzhou, China
5 shared publications - Haojian Lu
Department of Control Science and Engineering, Zhejiang University, Hangzhou, China
3 shared publications - Huan Yin
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong
3 shared publications - Xieyuanli Chen
National university of defense technology, Changsha, China
3 shared publications
Latest Publications
- 29 April 2026
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 11  (113) , eaea2092
A retrieval-augmented framework enabling VLM spatial awareness for object-centric robot manipulation
Connecting the semantic reasoning of vision-language models (VLMs) to the precise geometric demands of robotic manipulation remains a fundamental challenge. Although VLMs can interpret high-level commands, they lack the intrinsic spatial intelligence required for tasks demanding precise object placement, orientation, and physical reasoning. Here, we introduce Retrieval-Augmented Manipulation (RAM), an object-centric framework that endows general-purpose vision foundation models with the spatial reasoning necessary for robust manipulation. RAM bridges the semantic-to-geometric gap by grounding abstract concepts into an explicit, object-centric three-dimensional (3D) representation. This grounded information is then provided as augmented context to the VLM, empowering it to decompose complex instructions into a sequence of spatially precise and physically plausible subgoals. We demonstrate that RAM, in a zero-shot setting on a real-world robot, can execute these subgoals to fulfill complex spatial language instructions, complete spatially aware manipulation under the guidance of a single 2D image, and adaptively replan tasks by reasoning about physical constraints like object size and collisions. Quantitative evaluations on the Common Object in 3D (CO3D) dataset also validated that RAM’s core vision module generalizes to previously unseen object categories and is robust to variations in shape and occlusions. By providing a structured bridge between semantic intent and geometric execution, RAM represents a critical step toward developing more physically intelligent and general-purpose robotic systems.
- 29 April 2026
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 11  (113) , eaea2092
A retrieval-augmented framework enabling VLM spatial awareness for object-centric robot manipulation
Connecting the semantic reasoning of vision-language models (VLMs) to the precise geometric demands of robotic manipulation remains a fundamental challenge. Although VLMs can interpret high-level commands, they lack the intrinsic spatial intelligence required for tasks demanding precise object placement, orientation, and physical reasoning. Here, we introduce Retrieval-Augmented Manipulation (RAM), an object-centric framework that endows general-purpose vision foundation models with the spatial reasoning necessary for robust manipulation. RAM bridges the semantic-to-geometric gap by grounding abstract concepts into an explicit, object-centric three-dimensional (3D) representation. This grounded information is then provided as augmented context to the VLM, empowering it to decompose complex instructions into a sequence of spatially precise and physically plausible subgoals. We demonstrate that RAM, in a zero-shot setting on a real-world robot, can execute these subgoals to fulfill complex spatial language instructions, complete spatially aware manipulation under the guidance of a single 2D image, and adaptively replan tasks by reasoning about physical constraints like object size and collisions. Quantitative evaluations on the Common Object in 3D (CO3D) dataset also validated that RAM’s core vision module generalizes to previously unseen object categories and is robust to variations in shape and occlusions. By providing a structured bridge between semantic intent and geometric execution, RAM represents a critical step toward developing more physically intelligent and general-purpose robotic systems.
- 2 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10  (6) , 5561-5568
Sparse Hierarchical LiDAR Bundle Adjustment for Online Collaborative Localization and Mapping
This letter presents a sparse hierarchical LiDAR bundle adjustment method for online multi-robot collaborative simultaneous localization and mapping (C-SLAM). The motivation behind this work is that the pose graph cannot directly reflect map inconsistencies. As a result, the map divergence across multiple robots persists even after pose graph optimization. While existing methods have utilized LiDAR bundle adjustment (BA) to address the divergence issues, none of these methods are particularly effective at improving online localization accuracy in multi-robot scenarios. In this work, we propose a sparse hierarchical mechanism where LiDAR bundle adjustment functions as a low-latency module within a centralized C-SLAM system. The hierarchical design accelerates the original BA process, while sparse selection further decreases the problem's complexity, thereby improving computational efficiency. The combination of high-frequency multi-robot pose graph optimization (MR-PGO) and low-frequency multi-robot hierarchical bundle adjustment (MR-HBA) improves accuracy and provides real-time localization results. To validate the effectiveness of our proposed method, we conduct comparative experiments using multiple public datasets and a self-collected dataset, benchmarking against state-of-the-art multi-robot SLAM systems. The results demonstrate that our method significantly enhances the accuracy of localization and mapping. Additionally, we have made the entire system available as an open-source implementation to benefit the broader research community.
- 2 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10  (6) , 5561-5568
Sparse Hierarchical LiDAR Bundle Adjustment for Online Collaborative Localization and Mapping
This letter presents a sparse hierarchical LiDAR bundle adjustment method for online multi-robot collaborative simultaneous localization and mapping (C-SLAM). The motivation behind this work is that the pose graph cannot directly reflect map inconsistencies. As a result, the map divergence across multiple robots persists even after pose graph optimization. While existing methods have utilized LiDAR bundle adjustment (BA) to address the divergence issues, none of these methods are particularly effective at improving online localization accuracy in multi-robot scenarios. In this work, we propose a sparse hierarchical mechanism where LiDAR bundle adjustment functions as a low-latency module within a centralized C-SLAM system. The hierarchical design accelerates the original BA process, while sparse selection further decreases the problem's complexity, thereby improving computational efficiency. The combination of high-frequency multi-robot pose graph optimization (MR-PGO) and low-frequency multi-robot hierarchical bundle adjustment (MR-HBA) improves accuracy and provides real-time localization results. To validate the effectiveness of our proposed method, we conduct comparative experiments using multiple public datasets and a self-collected dataset, benchmarking against state-of-the-art multi-robot SLAM systems. The results demonstrate that our method significantly enhances the accuracy of localization and mapping. Additionally, we have made the entire system available as an open-source implementation to benefit the broader research community.
- 18 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 41  (15523098) , 1861-1881
RING#: PR-By-PE Global Localization With Roto-Translation Equivariant Gram Learning
Global localization using onboard perception sensors, such as cameras and LiDARs, is crucial in autonomous driving and robotics applications when GPS signals are unreliable. Most approaches achieve global localization by sequential place recognition (PR) and pose estimation (PE). Some methods train separate models for each task, while others employ a single model with dual heads, trained jointly with separate task-specific losses. However, the accuracy of localization heavily depends on the success of place recognition, which often fails in scenarios with significant changes in viewpoint or environmental appearance. Consequently, this renders the final pose estimation of localization ineffective. To address this, we introduce a new paradigm, PR-by-PE localization, which bypasses the need for separate place recognition by directly deriving it from pose estimation. We propose RING#, an end-to-end PR-by-PE localization network that operates in the bird's-eye-view (BEV) space, compatible with both vision and LiDAR sensors. RING# incorporates a novel design that learns two equivariant representations from BEV features, enabling globally convergent and computationally efficient pose estimation. Comprehensive experiments on the NCLT and Oxford datasets show that RING# outperforms state-of-the-art methods in both vision and LiDAR modalities, validating the effectiveness of the proposed approach. The code is available at https://github.com/lus6-Jenny/RINGSharp.
- 18 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 41  (15523098) , 1861-1881
RING#: PR-By-PE Global Localization With Roto-Translation Equivariant Gram Learning
Global localization using onboard perception sensors, such as cameras and LiDARs, is crucial in autonomous driving and robotics applications when GPS signals are unreliable. Most approaches achieve global localization by sequential place recognition (PR) and pose estimation (PE). Some methods train separate models for each task, while others employ a single model with dual heads, trained jointly with separate task-specific losses. However, the accuracy of localization heavily depends on the success of place recognition, which often fails in scenarios with significant changes in viewpoint or environmental appearance. Consequently, this renders the final pose estimation of localization ineffective. To address this, we introduce a new paradigm, PR-by-PE localization, which bypasses the need for separate place recognition by directly deriving it from pose estimation. We propose RING#, an end-to-end PR-by-PE localization network that operates in the bird's-eye-view (BEV) space, compatible with both vision and LiDAR sensors. RING# incorporates a novel design that learns two equivariant representations from BEV features, enabling globally convergent and computationally efficient pose estimation. Comprehensive experiments on the NCLT and Oxford datasets show that RING# outperforms state-of-the-art methods in both vision and LiDAR modalities, validating the effectiveness of the proposed approach. The code is available at https://github.com/lus6-Jenny/RINGSharp.
- 2 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9  (6) , 5655-5662
Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments
Goal-reaching in unknown environments is one of the essential tasks in robot applications. Large-scale perception and long-horizon decision-making are the keys to solving this task as the operation scope expands or complexity rises. Existing navigation methods may suffer from degraded performance in complicated environments induced by scalability-limited map representation or greedy decision strategy. We propose the path-extended graph as a compact map representation providing sufficient structural information within a reasonable receptive field and incorporate it into a hierarchical policy for higher efficiency and generalizability. The path-extended graph contains the concise topology of environment structure and frontier layout for large-scale perception, avoiding the impact of redundant information. The hierarchical policy solves long-horizon non-myopic decision-making through a high-level frontier selection policy using deep reinforcement learning (DRL) and a low-level motion controller that handles path planning and collision avoidance. Simulation and real-world experiments demonstrate that our method outperforms other competitive approaches in avoiding redundant movement and achieves efficient goal-reaching, especially in complex environments.
- 2 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9  (6) , 5655-5662
Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments
Goal-reaching in unknown environments is one of the essential tasks in robot applications. Large-scale perception and long-horizon decision-making are the keys to solving this task as the operation scope expands or complexity rises. Existing navigation methods may suffer from degraded performance in complicated environments induced by scalability-limited map representation or greedy decision strategy. We propose the path-extended graph as a compact map representation providing sufficient structural information within a reasonable receptive field and incorporate it into a hierarchical policy for higher efficiency and generalizability. The path-extended graph contains the concise topology of environment structure and frontier layout for large-scale perception, avoiding the impact of redundant information. The hierarchical policy solves long-horizon non-myopic decision-making through a high-level frontier selection policy using deep reinforcement learning (DRL) and a low-level motion controller that handles path planning and collision avoidance. Simulation and real-world experiments demonstrate that our method outperforms other competitive approaches in avoiding redundant movement and achieves efficient goal-reaching, especially in complex environments.
- Open Access
A Survey on Global LiDAR Localization: Challenges, Advances and Open Problems
Knowledge about the own pose is key for all mobile robot applications. Thus pose estimation is part of the core functionalities of mobile robots. Over the last two decades, LiDAR scanners have become the standard sensor for robot localization and mapping. This article aims to provide an overview of recent progress and advancements in LiDAR-based global localization. We begin by formulating the problem and exploring the application scope. We then present a review of the methodology, including recent advancements in several topics, such as maps, descriptor extraction, and cross-robot localization. The contents of the article are organized under three themes. The first theme concerns the combination of global place retrieval and local pose estimation. The second theme is upgrading single-shot measurements to sequential ones for sequential global localization. Finally, the third theme focuses on extending single-robot global localization to cross-robot localization in multi-robot systems. We conclude the survey with a discussion of open challenges and promising directions in global LiDAR localization. To our best knowledge, this is the first comprehensive survey on global LiDAR localization for mobile robots.
Open AccessA Survey on Global LiDAR Localization: Challenges, Advances and Open Problems
Knowledge about the own pose is key for all mobile robot applications. Thus pose estimation is part of the core functionalities of mobile robots. Over the last two decades, LiDAR scanners have become the standard sensor for robot localization and mapping. This article aims to provide an overview of recent progress and advancements in LiDAR-based global localization. We begin by formulating the problem and exploring the application scope. We then present a review of the methodology, including recent advancements in several topics, such as maps, descriptor extraction, and cross-robot localization. The contents of the article are organized under three themes. The first theme concerns the combination of global place retrieval and local pose estimation. The second theme is upgrading single-shot measurements to sequential ones for sequential global localization. Finally, the third theme focuses on extending single-robot global localization to cross-robot localization in multi-robot systems. We conclude the survey with a discussion of open challenges and promising directions in global LiDAR localization. To our best knowledge, this is the first comprehensive survey on global LiDAR localization for mobile robots.
- Top Cited Papers
- 6 March 2024
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 132  (8) , 3139-3171
A Survey on Global LiDAR Localization: Challenges, Advances and Open Problems
- 6 March 2024
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 132  (8) , 3139-3171
Top Cited PapersA Survey on Global LiDAR Localization: Challenges, Advances and Open Problems
- 20 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45  (12) , 14366-14384
DPCN++: Differentiable Phase Correlation Network for Versatile Pose Registration
Pose registration is critical in vision and robotics. This article focuses on the challenging task of initialization-free pose registration up to 7DoF for homogeneous and heterogeneous measurements. While recent learning-based methods show promise using differentiable solvers, they either rely on heuristically defined correspondences or require initialization. Phase correlation seeks solutions in the spectral domain and is correspondence-free and initialization-free. Following this, we propose a differentiable solver and combine it with simple feature extraction networks, namely DPCN++. It can perform registration for homo/hetero inputs and generalizes well on unseen objects. Specifically, the feature extraction networks first learn dense feature grids from a pair of homogeneous/heterogeneous measurements. These feature grids are then transformed into a translation and scale invariant spectrum representation based on Fourier transform and spherical radial aggregation, decoupling translation and scale from rotation. Next, the rotation, scale, and translation are independently and efficiently estimated in the spectrum step-by-step. The entire pipeline is differentiable and trained end-to-end. We evaluate DCPN++ on a wide range of tasks taking different input modalities, including 2D bird’s-eye view images, 3D object and scene measurements, and medical images. Experimental results demonstrate that DCPN++ outperforms both classical and learning-based baselines, especially on partially observed and heterogeneous measurements.
- 20 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45  (12) , 14366-14384
DPCN++: Differentiable Phase Correlation Network for Versatile Pose Registration
Pose registration is critical in vision and robotics. This article focuses on the challenging task of initialization-free pose registration up to 7DoF for homogeneous and heterogeneous measurements. While recent learning-based methods show promise using differentiable solvers, they either rely on heuristically defined correspondences or require initialization. Phase correlation seeks solutions in the spectral domain and is correspondence-free and initialization-free. Following this, we propose a differentiable solver and combine it with simple feature extraction networks, namely DPCN++. It can perform registration for homo/hetero inputs and generalizes well on unseen objects. Specifically, the feature extraction networks first learn dense feature grids from a pair of homogeneous/heterogeneous measurements. These feature grids are then transformed into a translation and scale invariant spectrum representation based on Fourier transform and spherical radial aggregation, decoupling translation and scale from rotation. Next, the rotation, scale, and translation are independently and efficiently estimated in the spectrum step-by-step. The entire pipeline is differentiable and trained end-to-end. We evaluate DCPN++ on a wide range of tasks taking different input modalities, including 2D bird’s-eye view images, 3D object and scene measurements, and medical images. Experimental results demonstrate that DCPN++ outperforms both classical and learning-based baselines, especially on partially observed and heterogeneous measurements.
- Top Cited Papers
- 18 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 39  (6) , 4616-4635
RING++: Roto-Translation Invariant Gram for Global Localization on a Sparse Scan Map
Global localization plays a critical role in many robot applications. LiDAR-based global localization draws the community's focus with its robustness against illumination and seasonal changes. To further improve the localization under large viewpoint differences, we propose RING++ that has roto-translation-invariant representation for place recognition and global convergence for both rotation and translation estimation. With the theoretical guarantee, RING++ is able to address the large viewpoint difference using a lightweight map with sparse scans. In addition, we derive sufficient conditions of feature extractors for the representation preserving the roto-translation invariance, making RING++ a framework applicable to generic multichannel features. To the best of our knowledge, this is the first learning-free framework to address all the subtasks of global localization in the sparse scan map. Validations on real-world datasets show that our approach demonstrates better performance than state-of-the-art learning-free methods and competitive performance with learning-based methods. Finally, we integrate RING++ into a multirobot/session simultaneous localization and mapping system, performing its effectiveness in collaborative applications.
- 18 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 39  (6) , 4616-4635
Top Cited PapersRING++: Roto-Translation Invariant Gram for Global Localization on a Sparse Scan Map
Global localization plays a critical role in many robot applications. LiDAR-based global localization draws the community's focus with its robustness against illumination and seasonal changes. To further improve the localization under large viewpoint differences, we propose RING++ that has roto-translation-invariant representation for place recognition and global convergence for both rotation and translation estimation. With the theoretical guarantee, RING++ is able to address the large viewpoint difference using a lightweight map with sparse scans. In addition, we derive sufficient conditions of feature extractors for the representation preserving the roto-translation invariance, making RING++ a framework applicable to generic multichannel features. To the best of our knowledge, this is the first learning-free framework to address all the subtasks of global localization in the sparse scan map. Validations on real-world datasets show that our approach demonstrates better performance than state-of-the-art learning-free methods and competitive performance with learning-based methods. Finally, we integrate RING++ into a multirobot/session simultaneous localization and mapping system, performing its effectiveness in collaborative applications.
- Open Access
Learning Interpretable BEV Based VIO without Deep Neural Networks
Monocular visual-inertial odometry (VIO) is a critical problem in robotics and autonomous driving. Traditional methods solve this problem based on filtering or optimization. While being fully interpretable, they rely on manual interference and empirical parameter tuning. On the other hand, learning-based approaches allow for end-to-end training but require a large number of training data to learn millions of parameters. However, the non-interpretable and heavy models hinder the generalization ability. In this paper, we propose a fully differentiable, and interpretable, bird-eye-view (BEV) based VIO model for robots with local planar motion that can be trained without deep neural networks. Specifically, we first adopt Unscented Kalman Filter as a differentiable layer to predict the pitch and roll, where the covariance matrices of noise are learned to filter out the noise of the IMU raw data. Second, the refined pitch and roll are adopted to retrieve a gravity-aligned BEV image of each frame using differentiable camera projection. Finally, a differentiable pose estimator is utilized to estimate the remaining 3 DoF poses between the BEV frames: leading to a 5 DoF pose estimation. Our method allows for learning the covariance matrices end-to-end supervised by the pose estimation loss, demonstrating superior performance to empirical baselines. Experimental results on synthetic and real-world datasets demonstrate that our simple approach is competitive with state-of-the-art methods and generalizes well on unseen scenes.
Open AccessLearning Interpretable BEV Based VIO without Deep Neural Networks
Monocular visual-inertial odometry (VIO) is a critical problem in robotics and autonomous driving. Traditional methods solve this problem based on filtering or optimization. While being fully interpretable, they rely on manual interference and empirical parameter tuning. On the other hand, learning-based approaches allow for end-to-end training but require a large number of training data to learn millions of parameters. However, the non-interpretable and heavy models hinder the generalization ability. In this paper, we propose a fully differentiable, and interpretable, bird-eye-view (BEV) based VIO model for robots with local planar motion that can be trained without deep neural networks. Specifically, we first adopt Unscented Kalman Filter as a differentiable layer to predict the pitch and roll, where the covariance matrices of noise are learned to filter out the noise of the IMU raw data. Second, the refined pitch and roll are adopted to retrieve a gravity-aligned BEV image of each frame using differentiable camera projection. Finally, a differentiable pose estimator is utilized to estimate the remaining 3 DoF poses between the BEV frames: leading to a 5 DoF pose estimation. Our method allows for learning the covariance matrices end-to-end supervised by the pose estimation loss, demonstrating superior performance to empirical baselines. Experimental results on synthetic and real-world datasets demonstrate that our simple approach is competitive with state-of-the-art methods and generalizes well on unseen scenes.
- Open Access
- 1 January 2022
- Published by National Library of Serbia in Thermal Science
- Vol. 26  (4 Part B) , 3359-3371
Thermal analysis on natural-convection coupled with radiative heat transfer in a saturated porous cavity
Porous foam is an ideal material for enhancing radiative heat transfer in numerous thermal equipment. The solid skeletons of porous foams can absorb/ release radiative energy and transfer convective energy with the surrounding fluid in the pores. In this paper, the conduction-convection-radiation coupling heat transfer in a porous cavity is investigated. A local thermal nonequilibrium model is used to represent the energy transport during the solid and fluid phases. The heat flux caused by thermal radiation is obtained by solving the radiation transfer equation. The thermal and fluid fields are studied to discern various parameters, including the Planck numbers Pl , the modified Rayleigh numbers Ra , and the interphase heat transfer coefficients H . Our study indicates the following: (1) the effect of radiation can be neglected when Pl > 20; (2) the modified Rayleigh numbers have little influence on the solid temperature when the radiative heat transfer is dominant and the convective heat transfer between the two phases is weak; and (3) the local thermal-equilibrium can be formed when H exhibits high values.
- 1 January 2022
- Published by National Library of Serbia in Thermal Science
- Vol. 26  (4 Part B) , 3359-3371
Open AccessThermal analysis on natural-convection coupled with radiative heat transfer in a saturated porous cavity
Porous foam is an ideal material for enhancing radiative heat transfer in numerous thermal equipment. The solid skeletons of porous foams can absorb/ release radiative energy and transfer convective energy with the surrounding fluid in the pores. In this paper, the conduction-convection-radiation coupling heat transfer in a porous cavity is investigated. A local thermal nonequilibrium model is used to represent the energy transport during the solid and fluid phases. The heat flux caused by thermal radiation is obtained by solving the radiation transfer equation. The thermal and fluid fields are studied to discern various parameters, including the Planck numbers Pl , the modified Rayleigh numbers Ra , and the interphase heat transfer coefficients H . Our study indicates the following: (1) the effect of radiation can be neglected when Pl > 20; (2) the modified Rayleigh numbers have little influence on the solid temperature when the radiative heat transfer is dominant and the convective heat transfer between the two phases is weak; and (3) the local thermal-equilibrium can be formed when H exhibits high values.