TSTianyu Sun

Tianyu Sun

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Tianyu Sun has a total of 38 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Uncertainty-Aware Laser Stripe Segmentation With Nonlocal Mechanisms for Welding Robots

    by Yixiang Dai, Siang Chen, Tianyu Sun, Zimo Fan, Chun Zhang, Xiaobing Feng, Guijin Wang

    The line-structured-light system has been widely adopted for weld seam reconstruction and tracking in intelligent welding robots. However, extracting projected laser stripes from captured images remains a significant challenge due to the presence of intense noises in welding environments. In this work, we propose an uncertainty-aware non-local laser stripe segmentation network(UNLS-Net) to achieve precise laser stripe extraction under real-world, challenging welding conditions. The proposed framework designs an uncertainty-aware policy that refines coarse segmentation results using combined epistemic-aleatoric uncertainty maps. Additionally, non-local attention modules are incorporated to enhance spatial correlation, thereby preserving the continuity of laser stripes. Comprehensive experiments are conducted on our large-scale, shape-diverse laser stripe dataset comprising 3,136 welding images with varying weld seam geometries, sizes, and noise profiles. The proposed method demonstrates superior performance compared to existing segmentation approaches, achieving significant improvements in both laser stripe continuity and denoising effectiveness.


  • Variation-Robust Few-Shot 3D Affordance Segmentation for Robotic Manipulation

    by Dingchang Hu, Tianyu Sun, Pengwei Xie, Siang Chen, Huazhong Yang, Guijin Wangy

    Traditional affordance segmentation on 3D point cloud objects requires massive amounts of annotated training data and can only make predictions within predefined classes and affordance tasks. To overcome these limitations, we propose a variation-robust few-shot 3D affordance segmentation network (VRNet) for robotic manipulation, which requires only several affordance annotations for novel object classes and manipulation tasks. In particular, we design an orientation-tolerant feature extractor to address pose variation between support and query point cloud objects, and present a multi-scale label propagation algorithm for variation in completeness. Extensive experiments on affordance datasets show that VRNet provides the best segmentation performance compared with previous works. Moreover, experiments in real robotic scenarios demonstrate the generalization ability of our method.


  • Segment, Lift and Fit: Automatic 3D Shape Labeling from 2D Prompts

    by Jianhao Li, Tianyu Sun, Zhongdao Wang, Enze Xie, Bailan Feng, Hongbo Zhang, Ze Yuan, Ke Xu, Jiaheng Liu, Ping Luo


  • Diffusion-Based Depth Inpainting for Transparent and Reflective Objects

    by Tianyu Sun, Dingchang Hu, Yixiang Dai, Guijin Wang

    Transparent and reflective objects, which are common in our everyday lives, present a significant challenge to 3D imaging techniques due to their unique visual and optical properties. Faced with these types of objects, RGB-D cameras fail to capture the real depth value with their accurate spatial information. To address this issue, we propose DITR, a diffusion-based Depth Inpainting framework specifically designed for Transparent and Reflective objects. This network consists of two stages, including a Region Proposal stage and a Depth Inpainting stage. DITR dynamically analyzes the optical and geometric depth loss and inpaints them automatically. Furthermore, comprehensive experimental results demonstrate that DITR is highly effective in depth inpainting tasks of transparent and reflective objects with robust adaptability.


  • Open Access

    Segment, Lift and Fit: Automatic 3D Shape Labeling from 2D Prompts

    by Jianhao Li, Tianyu Sun, Zhongdao Wang, Enze Xie, Bailan Feng, Hongbo Zhang, Ze Yuan, Ke Xu, Jiaheng Liu, Ping Luo

    This paper proposes an algorithm for automatically labeling 3D objects from 2D point or box prompts, especially focusing on applications in autonomous driving. Unlike previous arts, our auto-labeler predicts 3D shapes instead of bounding boxes and does not require training on a specific dataset. We propose a Segment, Lift, and Fit (SLF) paradigm to achieve this goal. Firstly, we segment high-quality instance masks from the prompts using the Segment Anything Model (SAM) and transform the remaining problem into predicting 3D shapes from given 2D masks. Due to the ill-posed nature of this problem, it presents a significant challenge as multiple 3D shapes can project into an identical mask. To tackle this issue, we then lift 2D masks to 3D forms and employ gradient descent to adjust their poses and shapes until the projections fit the masks and the surfaces conform to surrounding LiDAR points. Notably, since we do not train on a specific dataset, the SLF auto-labeler does not overfit to biased annotation patterns in the training set as other methods do. Thus, the generalization ability across different datasets improves. Experimental results on the KITTI dataset demonstrate that the SLF auto-labeler produces high-quality bounding box annotations, achieving an AP@0.5 IoU of nearly 90\%. Detectors trained with the generated pseudo-labels perform nearly as well as those trained with actual ground-truth annotations. Furthermore, the SLF auto-labeler shows promising results in detailed shape predictions, providing a potential alternative for the occupancy annotation of dynamic objects.

  • Part-Guided 3D RL for Sim2Real Articulated Object Manipulation

    by Pengwei Xie, Rui Chen, Siang Chen, Yuzhe Qin, Fanbo Xiang, Tianyu Sun, Jing Xu, Guijin Wang, Hao Su

    Manipulating unseen articulated objects through visual feedback is a critical but challenging task for real robots. Existing learning-based solutions mainly focus on visual affordance learning or other pre-trained visual models to guide manipulation policies, which face challenges for novel instances in real-world scenarios. In this letter, we propose a novel part-guided 3D RL framework, which can learn to manipulate articulated objects without demonstrations. We combine the strengths of 2D segmentation and 3D RL to improve the efficiency of RL policy training. To improve the stability of the policy on real robots, we design a Frame-consistent Uncertainty-aware Sampling (FUS) strategy to get a condensed and hierarchical 3D representation. In addition, a single versatile RL policy can be trained on multiple articulated object manipulation tasks simultaneously in simulation and shows great generalizability to novel categories and instances. Experimental results demonstrate the effectiveness of our framework in both simulation and real-world settings.


  • TROSD: A New RGB-D Dataset for Transparent and Reflective Object Segmentation in Practice

    by Tianyu Sun, Guodong Zhang, Wenming Yang, Jing-Hao Xue, Guijin Wang

    Transparent and reflective objects are omnipresent in our daily life, but their unique visual and optical characteristics are notoriously challenging even for state-of-the-art deep networks of semantic segmentation. To alleviate this challenge, we construct a new large-scale real-world RGB-D dataset called TROSD, which is more comprehensive than existing datasets for transparent and reflective object segmentation. Our TROSD dataset contains 11,060 RGB-D images with three semantic classes in terms of transparent objects, reflective objects, and others, covering a variety of daily scenes. Together with the dataset, we also introduce a novel network (TROSNet) as a high-standard baseline to assist other researchers to develop and benchmark their algorithms of transparent and reflective object segmentation. Moreover, extensive experiments also clearly show that the proposed TROSD dataset has an excellent capacity to facilitate the development of semantic segmentation algorithms with strong generalizability.