MagiClaw: A Dual-Use, Vision-Based Soft Gripper for Bridging the Human Demonstration to Robotic Deployment Gap
Preprint
Abstract
The transfer of manipulation skills from human demonstration to robotic execution is often hindered by a "domain gap" in sensing and morphology. This paper introduces MagiClaw, a versatile two-finger end-effector designed to bridge this gap. MagiClaw functions interchangeably as both a handheld tool for intuitive data collection and a robotic end-effector for policy deployment, ensuring hardware consistency and reliability. Each finger incorporates a Soft Polyhedral Network (SPN) with an embedded camera, enabling vision-based estimation of 6-DoF forces and contact deformation. This proprioceptive data is fused with exteroceptive environmental sensing from an integrated iPhone, which provides 6D pose, RGB video, and LiDAR-based depth maps. Through a custom iOS application, MagiClaw streams synchronized, multi-modal data for real-time teleoperation, offline policy learning, and immersive control via mixed-reality interfaces. We demonstrate how this unified system architecture lowers the barrier to collecting high-fidelity, contact-rich datasets and accelerates the development of generalizable manipulation policies. Please refer to the iOS app at https://apps.apple.com/cn/app/magiclaw/id6661033548 for further details.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulation2025
- Tactile sensors: A reviewMeasurement, 2024
- A Survey of Imitation Learning: Algorithms, Recent Developments, and ChallengesIEEE Transactions on Cybernetics, 2024
- Visual-Force-Tactile Fusion for Gentle Intricate Insertion TasksIEEE Robotics and Automation Letters, 2024
- Recent Advances in Robot Learning from DemonstrationAnnual Review of Control, Robotics, and Autonomous Systems, 2020
- Trends and challenges in robot manipulationScience, 2019
- Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collectionThe International Journal of Robotics Research, 2017
- A survey of robot learning from demonstrationRobotics and Autonomous Systems, 2009
- Robot Programming by Demonstration2008
- Tactile Sensing: New Directions, New ChallengesThe International Journal of Robotics Research, 2000