JLJin Liu

Jin Liu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jin Liu has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • HIVE-6D: Hierarchical Visual-Tactile In-Hand Pose Estimation and Tracking

    by Jin Liu, Weihao Wang, Hongzhan Yu, Can Zhao, Haonan Zhao, Daolin Ma, Weiming Wang

  • Multi-tactile sensor calibration via motion constraints with tactile measurements

    by Hexi Yu, Jin Liu, Daolin Ma

    This paper addresses a fundamental problem in multi-finger robot dexterous manipulation, Multi-Tactile Sensor Calibration (MTSC). It involves estimating the relative poses between multiple tactile sensors using their intrinsic measurements, crucial for coordinating multiple fingers in a dexterous robotic hand, especially when accurate encoders are unavailable. Unlike conventional multi-sensor calibration methods (like cameras), which rely on overlapping sensing regions and feature point matching, calibrating multiple tactile sensors presents unique challenges because these sensors cannot have overlapping sensing regions, precluding the use of shared key-points visible from different sensors. In this paper, we establish the theoretical basis for the MTSC problem, constructing constraint equations based on motions measured at multiple locations on the same grasped object from different tactile sensors. The key is that all these measurements correspond to a shared rigid body motion. A calibration scheme is proposed accordingly, with theoretical analysis conducted to enhance precision. Results in simulation, with objects of different shapes, confirm the validity of the proposed calibration approach. Furthermore, calibration of the relative pose between two GelSlim vision-based tactile sensors in real experiments demonstrates good agreement with the ground truth. The proposed theory and method not only shed light on explaining the accuracy gap between proprioception and tactile sensing in multi-finger manipulation but also pave the way for tactile-encoder-mixed or encoder-free solutions in dexterous multi-finger coordination.


  • iFEM2.0: Dense 3-D Contact Force Field Reconstruction and Assessment for Vision-Based Tactile Sensors

    by Can Zhao, Jin Liu, Daolin Ma

    Vision-based tactile sensors offer rich tactile information through high-resolution tactile images, enabling the reconstruction of dense contact force fields on the sensor surface. However, accurately reconstructing the 3-D contact force distribution remains a challenge. In this article, we propose the multilayer inverse finite-element method (iFEM2.0) as a robust and generalized approach to reconstruct dense contact force distribution. We systematically analyze various parameters within the iFEM2.0 framework, and determine the appropriate parameter combinations through simulation and in situ mechanical calibration. Our approach incorporates multilayer mesh constraints and ridge regularization to enhance robustness. Furthermore, as no off-the-shelf measurement equipment or criterion metrics exist for 3-D contact force distribution perception, we present a benchmark covering accuracy, fidelity, and noise resistance that can serve as a cornerstone for other future force distribution reconstruction methods. The proposed iFEM2.0 demonstrates good performance in both simulation- and experiment-based evaluations. Such dense 3-D contact force information is critical for enabling dexterous robotic manipulation that handles both rigid and soft materials.


  • CPPF++: Uncertainty-Aware Sim2Real Object Pose Estimation by Vote Aggregation

    by Yang You, Wenhao He, Jin Liu, Hongkai Xiong, Weiming Wang, Cewu Lu

    Object pose estimation constitutes a critical area within the domain of 3D vision. While contemporary state-of-the-art methods that leverage real-world pose annotations have demonstrated commendable performance, the procurement of such real training data incurs substantial costs. This paper focuses on a specific setting wherein only 3D CAD models are utilized as a priori knowledge, devoid of any background or clutter information. We introduce a novel method, CPPF++, designed for sim-to-real category-level pose estimation. This method builds upon the foundational point-pair voting scheme of CPPF, reformulating it through a probabilistic view. To address the challenge posed by vote collision, we propose a novel approach that involves modeling the voting uncertainty by estimating the probabilistic distribution of each point pair within the canonical space. Furthermore, we augment the contextual information provided by each voting unit through the introduction of $N$ -point tuples. To enhance the robustness and accuracy of the model, we incorporate several innovative modules, including noisy pair filtering, online alignment optimization, and a tuple feature ensemble. Alongside these methodological advancements, we introduce a new category-level pose estimation dataset, named DiversePose 300. Empirical evidence demonstrates that our method significantly surpasses previous sim-to-real approaches and achieves comparable or superior performance on novel datasets.


  • Real-Time Reconstruction of 3-D Tactile Motion Field via Multitask Learning

    by Jin Liu, Hexi Yu, Can Zhao, Wenhai Liu, Daolin Ma, Weiming Wang

    The 3-D motion field on the surface of the vision-based tactile sensor contains rich tactile information and serves as the foundation for many downstream tasks. However, achieving real-time and precise reconstruction of 3-D motion field is challenging. In this study, the 3-D motion field is decomposed into depth and 2-D motion field, and a multitask learning model is employed to simultaneously estimate both. The approach excels in accuracy and real-time performance (9.20 ms/frame). For depth reconstruction, the model achieves an average mean absolute error (MAE) of 0.062 mm. For 2-D motion field tracking, an effective structured marker tracking algorithm (SMTA) is introduced, and a marker tracking network (MaTnet) is constructed based on it. This network features dynamic sensing fields and a distance field (DF) auxiliary task, offering strong generalization, interpretability, and ease of training. It exhibits excellent tracking performance in various deformations, with an average error of 1.044 pixels (about 0.03 mm). Finally, the strong transferability of the multitask model is demonstrated, with a maximum decrease in depth reconstruction accuracy of 0.018 mm.