A Novel Visuo-Tactile Sensor for Contacting Perception and Object Recognition
- 8 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 26 (2) , 1739-1748
Abstract
High-performance sensing is essential for robotic hands because of their critical requirement for real-time and rich feedback information from both the operating target and the environment during intricate operations in challenging conditions. Among the few precepting principles for robots, Visuo-Tactile Sensors have attracted vast attention from both the academic community and the engineering field due to their excellent space effectiveness and resolution. This paper developed a novel Visuo-Tactile Sensor with a newly designed variable-transparency film, which offers the ability of both contact perception and object recognition without interference from the background light. To decouple the dual modalities of tactile and visual information, visual features from both non-contact and contact states are utilized in the perception algorithm. Moreover, the sensor could estimate the pose of the grasped object via visual recognition, eliminating the influence of ambient light. The proposed Visuo-Tactile Sensor underwent quantitative evaluation through anti-interference tests, in-hand object pose estimation tests, as well as underwater recognition and grasping tests. The experimental results indicate that the proposed sensor achieves a minimum recognition accuracy of 93.3% for complex-shaped objects, along with negligible estimation errors under specific working conditions. The proposed Visuo-Tactile Sensor utilizes a novel variable-transparency film, unlike GelSight or other existing soft-bubble-based tactile sensors. The proposed sensor features decoupled visuo-tactile perception under both bright and dark working conditions, exhibiting good robustness to avoid interference from background light. Anti-interference tests, in-hand object pose estimation tests, underwater recognition and grasp tests are conducted to prove the reliability, robustness, precision, effectiveness, and universality of the developed Visuo-Tactile Sensor under different working conditions.Keywords
Funding Information
- Natural Science Foundation of Zhejiang Province (LZ23E050005)
- Natural Science Foundation of Zhejiang Province (LMS25E050008)
This publication has 35 references indexed in Scilit:
- A soft thumb-sized vision-based sensor with accurate all-round force perceptionNature Machine Intelligence, 2022
- On the Design and Development of Vision-based Tactile SensorsJournal of Intelligent & Robotic Systems, 2021
- GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger2021
- Visuotactile Sensors With Emphasis on GelSight Sensor: A ReviewIEEE Sensors Journal, 2020
- Benchmarking Functionalities of Domestic Service Robots Through Scientific CompetitionsKI - Künstliche Intelligenz, 2019
- FilterReg: Robust and Efficient Probabilistic Point-Set Registration Using Gaussian Filter and Twist Parameterization2019
- Magnetic Resonance-Compatible Tactile Force Sensor Using Fiber Optics and Vision SensorIEEE Sensors Journal, 2013
- Retrographic sensing for the measurement of surface texture and shape2009
- An Object Profile Detection by a High Resolution Tactile Sensor Using an Optical Conductive Plate2005
- Pedobarographic Foot — Pressure Measurements and Their Applications1979