In the Wild Ungraspable Object Picking with Bimanual Nonprehensile Manipulation
Authors:
- 19 May 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5669-5676
Abstract
Picking diverse objects in the real world is a fundamental robotics skill. However, many objects in such settings are bulky, heavy, or irregularly shaped, making them ungraspable by conventional end effectors like suction grippers and parallel jaw grippers (PJGs). In this paper, we expand the range of pickable items without hardware modifications using bimanual nonprehensile manipulation. We focus on a grocery shopping scenario, where a bimanual mobile manipulator equipped with a suction gripper and a PJG is tasked with re-trieving ungraspable items from tightly packed grocery shelves. From visual observations, our method first identifies optimal grasp points based on force closure and friction constraints. If the grasp points are occluded, a series of nonprehensile nudging motions are performed to clear the obstruction. A bimanual grasp utilizing contacts on the side of the end effectors is then executed to grasp the target item. In our replica grocery store, we achieved a 90% success rate over 102 trials in uncluttered scenes, and a 67 % success rate over 45 trials in cluttered scenes. We also deployed our system to a real-world grocery store and successfully picked previously unseen items. Our results highlight the potential of bimanual nonprehensile manipulation for in-the-wild robotic picking tasks. A video summarizing this work can be found at youtu. be/gOhOrDuK8jM.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Tasks2024
- Learning to Grasp in Clutter with Interactive Visual Failure Prediction2024
- Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes2021
- A Survey on Learning-Based Robotic GraspingCurrent Robotics Reports, 2020
- Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a reviewArtificial Intelligence Review, 2020
- Robust object grasping in clutter via singulation2019
- Analyzing at-home prosthesis use in unilateral upper-limb amputees to inform treatment & device design2017
- Dual arm manipulation—A surveyRobotics and Autonomous Systems, 2012
- A Planning Framework for Non-Prehensile Manipulation under Clutter and UncertaintyAutonomous Robots, 2012
- Advanced Bimanual Manipulation2012