Hao Xu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hao Xu has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Chi-Wing Fu
Chinese University of Hong Kong, Hong Kong, China
7 shared publications - Shuaicheng Liu
University of Electronic Science and Technology of China
5 shared publications - Haipeng Li
School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China
4 shared publications - Yinqiao Wang
Chinese University of Hong Kong, Hong Kong, China
4 shared publications - Guanghui Liu
University of Electronic Science and Technology of China
2 shared publications - Kunming Luo
Hong Kong University of Science and Technology, Hong Kong, China
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (6) , 1-12
Open-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstruction
We ask whether everyday open-world monocular videos can be turned into reusable 4D interaction primitives: articulated hand motion, object shape with 6D pose over time, and the when/where of contact. Such a capability would enable scalable mining of real interactions and, beyond reconstruction, support scene-aware synthesis and planning. However, reconstructing hand-object interaction (HOI) from challenging monocular videos remains difficult: methods often assume known objects or curated scenes, and separately estimated hands and objects easily become misaligned under clutter, occlusion, and unseen object geometries. Targeting this setting, we present Open-CHOIR, an Open-world Contact-aware HOI Reconstruction framework for a monocular camera, using contact as an explicit coupling signal between hands and objects. Open-CHOIR first initializes a coarse, contact-agnostic 4D HOI sequence from open-world visual priors. It then introduces a generative HOI spatial rectification module to predict ray-depth corrections and rectify hand-object relative placement, then derive initial per-frame contact correspondences on the rectified geometry. Last, a contact-aware joint optimization with dynamically updated contact constraints enforces geometric, temporal, and contact consistency. Experiments on controlled and challenging videos show that Open-CHOIR improves object reconstruction, physical plausibility, and temporal consistency over state-of-the-art methods. Code, data, and pretrained weights are available at https://github.com/hxwork/CHOIR.
- 9 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (6) , 1-12
Open AccessOpen-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstruction
We ask whether everyday open-world monocular videos can be turned into reusable 4D interaction primitives: articulated hand motion, object shape with 6D pose over time, and the when/where of contact. Such a capability would enable scalable mining of real interactions and, beyond reconstruction, support scene-aware synthesis and planning. However, reconstructing hand-object interaction (HOI) from challenging monocular videos remains difficult: methods often assume known objects or curated scenes, and separately estimated hands and objects easily become misaligned under clutter, occlusion, and unseen object geometries. Targeting this setting, we present Open-CHOIR, an Open-world Contact-aware HOI Reconstruction framework for a monocular camera, using contact as an explicit coupling signal between hands and objects. Open-CHOIR first initializes a coarse, contact-agnostic 4D HOI sequence from open-world visual priors. It then introduces a generative HOI spatial rectification module to predict ray-depth corrections and rectify hand-object relative placement, then derive initial per-frame contact correspondences on the rectified geometry. Last, a contact-aware joint optimization with dynamically updated contact constraints enforces geometric, temporal, and contact consistency. Experiments on controlled and challenging videos show that Open-CHOIR improves object reconstruction, physical plausibility, and temporal consistency over state-of-the-art methods. Code, data, and pretrained weights are available at https://github.com/hxwork/CHOIR.
- Open Access
- 2 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-13
ATOM: Geometry-Aware Microgesture towards Object-Agnostic Tangible Interaction
- 2 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-13
Open AccessATOM: Geometry-Aware Microgesture towards Object-Agnostic Tangible Interaction
- Open Access
- 14 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Hierarchical Neural Semantic Representation for 3D Semantic Correspondence
- 14 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Open AccessHierarchical Neural Semantic Representation for 3D Semantic Correspondence
- 8 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (12) , 11201-11219
HandBooster+: Boosting 3D Hand-Mesh Reconstruction From Data Synthesis to Progressive Multi-Hypothesis Aggregation
Robustly reconstructing 3D hand mesh from a single image is very challenging, due to (i) the lack of diversity in existing real-world datasets and (ii) the ambiguity in occluded hand regions. While data synthesis helps relieve issue (i), the syn-to-real gap still hinders its usage. For issue (ii), most previous works produce deterministic results while other probabilistic methods rely on ground truths to choose the best hypothesis. In this work, we explore the diffusion model to alleviate these problems by collectively considering two perspectives: (i) conditional synthesis and sampling approach for realistic data generation and (ii) probabilistic modeling with progressive multi-hypothesis aggregation. First, we present HandBooster, a new approach to uplift the data diversity by training a conditional generative space on hand-object interactions and sampling the space to synthesize effective data with reliable 3D annotations and diverse hand appearances, poses, views, and backgrounds. Second, we design HandBooster+, a probabilistic diffusion-based model to further boost the 3D hand-mesh reconstruction performance by progressively aggregating the multiple hypotheses. Extensive experimental results show that our method significantly improves several baselines and achieves SOTA on the HO3D and DexYCB benchmarks.
- 8 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (12) , 11201-11219
HandBooster+: Boosting 3D Hand-Mesh Reconstruction From Data Synthesis to Progressive Multi-Hypothesis Aggregation
Robustly reconstructing 3D hand mesh from a single image is very challenging, due to (i) the lack of diversity in existing real-world datasets and (ii) the ambiguity in occluded hand regions. While data synthesis helps relieve issue (i), the syn-to-real gap still hinders its usage. For issue (ii), most previous works produce deterministic results while other probabilistic methods rely on ground truths to choose the best hypothesis. In this work, we explore the diffusion model to alleviate these problems by collectively considering two perspectives: (i) conditional synthesis and sampling approach for realistic data generation and (ii) probabilistic modeling with progressive multi-hypothesis aggregation. First, we present HandBooster, a new approach to uplift the data diversity by training a conditional generative space on hand-object interactions and sampling the space to synthesize effective data with reliable 3D annotations and diverse hand appearances, poses, views, and backgrounds. Second, we design HandBooster+, a probabilistic diffusion-based model to further boost the 3D hand-mesh reconstruction performance by progressively aggregating the multiple hypotheses. Extensive experimental results show that our method significantly improves several baselines and achieves SOTA on the HO3D and DexYCB benchmarks.
- Open Access
- 27 July 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 44 (4) , 1-16
Hand-Shadow Poser
Hand shadow art is a captivating art form, creatively using hand shadows to reproduce expressive shapes on the wall. In this work, we study an inverse problem: given a target shape, find the poses of left and right hands that together best produce a shadow resembling the input. This problem is nontrivial, since the design space of 3D hand poses is huge while being restrictive due to anatomical constraints. Also, we need to attend to the input's shape and crucial features, though the input is colorless and textureless. To meet these challenges, we design Hand-Shadow Poser, a three-stage pipeline, to decouple the anatomical constraints (by hand) and semantic constraints (by shadow shape): (i) a generative hand assignment module to explore diverse but reasonable left/right-hand shape hypotheses; (ii) a generalized hand-shadow alignment module to infer coarse hand poses with a similarity-driven strategy for selecting hypotheses; and (iii) a shadow-feature-aware refinement module to optimize the hand poses for physical plausibility and shadow feature preservation. Further, we design our pipeline to be trainable on generic public hand data, thus avoiding the need for any specialized training dataset. For method validation, we build a benchmark of 210 diverse shadow shapes of varying complexity and a comprehensive set of metrics, including a novel DINOv2-based evaluation metric. Through extensive comparisons with multiple baselines and user studies, our approach is demonstrated to effectively generate bimanual hand poses for a large variety of hand shapes for over 85% of the benchmark cases.
- 27 July 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 44 (4) , 1-16
Open AccessHand-Shadow Poser
Hand shadow art is a captivating art form, creatively using hand shadows to reproduce expressive shapes on the wall. In this work, we study an inverse problem: given a target shape, find the poses of left and right hands that together best produce a shadow resembling the input. This problem is nontrivial, since the design space of 3D hand poses is huge while being restrictive due to anatomical constraints. Also, we need to attend to the input's shape and crucial features, though the input is colorless and textureless. To meet these challenges, we design Hand-Shadow Poser, a three-stage pipeline, to decouple the anatomical constraints (by hand) and semantic constraints (by shadow shape): (i) a generative hand assignment module to explore diverse but reasonable left/right-hand shape hypotheses; (ii) a generalized hand-shadow alignment module to infer coarse hand poses with a similarity-driven strategy for selecting hypotheses; and (iii) a shadow-feature-aware refinement module to optimize the hand poses for physical plausibility and shadow feature preservation. Further, we design our pipeline to be trainable on generic public hand data, thus avoiding the need for any specialized training dataset. For method validation, we build a benchmark of 210 diverse shadow shapes of varying complexity and a comprehensive set of metrics, including a novel DINOv2-based evaluation metric. Through extensive comparisons with multiple baselines and user studies, our approach is demonstrated to effectively generate bimanual hand poses for a large variety of hand shapes for over 85% of the benchmark cases.
- 29 October 2024
- Published by Springer Nature
- p. 272-289
PointRegGPT: Boosting 3D Point Cloud Registration Using Generative Point-Cloud Pairs for Training
- 29 October 2024
- Published by Springer Nature
- p. 272-289
PointRegGPT: Boosting 3D Point Cloud Registration Using Generative Point-Cloud Pairs for Training
- Open Access
PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training
Data plays a crucial role in training learning-based methods for 3D point cloud registration. However, the real-world dataset is expensive to build, while rendering-based synthetic data suffers from domain gaps. In this work, we present PointRegGPT, boosting 3D point cloud registration using generative point-cloud pairs for training. Given a single depth map, we first apply a random camera motion to re-project it into a target depth map. Converting them to point clouds gives a training pair. To enhance the data realism, we formulate a generative model as a depth inpainting diffusion to process the target depth map with the re-projected source depth map as the condition. Also, we design a depth correction module to alleviate artifacts caused by point penetration during the re-projection. To our knowledge, this is the first generative approach that explores realistic data generation for indoor point cloud registration. When equipped with our approach, several recent algorithms can improve their performance significantly and achieve SOTA consistently on two common benchmarks. The code and dataset will be released on https://github.com/Chen-Suyi/PointRegGPT.
Open AccessPointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training
Data plays a crucial role in training learning-based methods for 3D point cloud registration. However, the real-world dataset is expensive to build, while rendering-based synthetic data suffers from domain gaps. In this work, we present PointRegGPT, boosting 3D point cloud registration using generative point-cloud pairs for training. Given a single depth map, we first apply a random camera motion to re-project it into a target depth map. Converting them to point clouds gives a training pair. To enhance the data realism, we formulate a generative model as a depth inpainting diffusion to process the target depth map with the re-projected source depth map as the condition. Also, we design a depth correction module to alleviate artifacts caused by point penetration during the re-projection. To our knowledge, this is the first generative approach that explores realistic data generation for indoor point cloud registration. When equipped with our approach, several recent algorithms can improve their performance significantly and achieve SOTA consistently on two common benchmarks. The code and dataset will be released on https://github.com/Chen-Suyi/PointRegGPT.