CZCan Zhao

Can Zhao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Can Zhao has a total of 18 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • HIVE-6D: Hierarchical Visual-Tactile In-Hand Pose Estimation and Tracking

    by Jin Liu, Weihao Wang, Hongzhan Yu, Can Zhao, Haonan Zhao, Daolin Ma, Weiming Wang

  • TACSEM: Tactile Sensing, 3-D State Estimation and In-Hand Manipulation of a Soft Hand

    by Hui Zhang, Qingyu Wang, Aiguo Song, Jianwei Lai, Daolin Ma, Can Zhao, Ge Gao, Qingdu Li, Jianwei Zhang

    In this article, we present a pneumatic soft hand with integrated tactile sensing, state estimation, and embodied intelligence capabilities. A novel state estimation method is proposed based on the linear elasticity theory. Following the theoretical principles above, the hardware for the soft hand is developed, which includes the fabrication of soft fingers with multiple sensors and the design of controller circuits. The soft hand achieves real-time tactile sensing for all three fingers at 7 Hz. A multimodal state estimation neural network is proposed to learn the principles of state estimation. It estimates the 3-D finger state with a 1.73 mm average error in 4.21 ms when deployed on the soft hand controller with a Jetson Orin Nano.Embodied intelligence in the soft hand is realized through vision-language models using a hierarchical pyramid prompt architecture, enabling the hand to perform in-hand manipulations using only the movements of its soft fingers, achieving an average success rate of 92.5%. The prompts, 3-D models, PCB images, source code, and video demos are provided in the supplemental materials attached to this article.


  • Tactile-Driven Dexterous In-Hand Writing via Extrinsic Contact Sensing

    by Can Zhao, Lingzi Xie, Bidan Huang, Shuai Wang, Daolin Ma

    Dexterous in-hand manipulation, especially involving interactions between grasped objects and external environments, remains a formidable challenge in robotics. This study tackles the complexities of in-hand manipulation under extrinsic contact through a representative three-finger handwriting task. We propose a hybrid arm-hand coordination framework that combines reinforcement learning with compliance control, offering both flexibility and robustness. Leveraging tactile sensors embedded in each finger, our tactile-driven estimation model dynamically predicts in-hand object pose and external contact, eliminating the need for fixed contact states. The proposed framework is first validated in simulation, where it successfully executes diverse writing tasks with accurate contact sensing. Sim-to-Real transfer is achieved through systematic calibration of finger joints and tactile sensors, supported by domain randomization. Real-world experiments further demonstrate the system's adaptability to writing tools with varying physical properties—such as radius, length, mass, and friction—while maintaining stability across different trajectories. Also see https://inhandwriting.github.io/. This work advances robotic manipulation capabilities in unstructured environments.


  • iFEM2.0: Dense 3-D Contact Force Field Reconstruction and Assessment for Vision-Based Tactile Sensors

    by Can Zhao, Jin Liu, Daolin Ma

    Vision-based tactile sensors offer rich tactile information through high-resolution tactile images, enabling the reconstruction of dense contact force fields on the sensor surface. However, accurately reconstructing the 3-D contact force distribution remains a challenge. In this article, we propose the multilayer inverse finite-element method (iFEM2.0) as a robust and generalized approach to reconstruct dense contact force distribution. We systematically analyze various parameters within the iFEM2.0 framework, and determine the appropriate parameter combinations through simulation and in situ mechanical calibration. Our approach incorporates multilayer mesh constraints and ridge regularization to enhance robustness. Furthermore, as no off-the-shelf measurement equipment or criterion metrics exist for 3-D contact force distribution perception, we present a benchmark covering accuracy, fidelity, and noise resistance that can serve as a cornerstone for other future force distribution reconstruction methods. The proposed iFEM2.0 demonstrates good performance in both simulation- and experiment-based evaluations. Such dense 3-D contact force information is critical for enabling dexterous robotic manipulation that handles both rigid and soft materials.


  • Real-Time Reconstruction of 3-D Tactile Motion Field via Multitask Learning

    by Jin Liu, Hexi Yu, Can Zhao, Wenhai Liu, Daolin Ma, Weiming Wang

    The 3-D motion field on the surface of the vision-based tactile sensor contains rich tactile information and serves as the foundation for many downstream tasks. However, achieving real-time and precise reconstruction of 3-D motion field is challenging. In this study, the 3-D motion field is decomposed into depth and 2-D motion field, and a multitask learning model is employed to simultaneously estimate both. The approach excels in accuracy and real-time performance (9.20 ms/frame). For depth reconstruction, the model achieves an average mean absolute error (MAE) of 0.062 mm. For 2-D motion field tracking, an effective structured marker tracking algorithm (SMTA) is introduced, and a marker tracking network (MaTnet) is constructed based on it. This network features dynamic sensing fields and a distance field (DF) auxiliary task, offering strong generalization, interpretability, and ease of training. It exhibits excellent tracking performance in various deformations, with an average error of 1.044 pixels (about 0.03 mm). Finally, the strong transferability of the multitask model is demonstrated, with a maximum decrease in depth reconstruction accuracy of 0.018 mm.