Grasp Generation with Depth Estimation from Color Images
Authors:
- Nguyen Van ThiepNTNguyen Van Thiep
- Vu Van DucVDVu Van Duc
- Hoang Ngoc AnhHAHoang Ngoc Anh
- Nguyen Thu UyenNUNguyen Thu Uyen
- Vu Duy QuangVQVu Duy Quang
- Tran Duc ThanhTTTran Duc Thanh
- Phan Khanh ToanPTPhan Khanh Toan
- Mai Truong AnhMAMai Truong Anh
- Duong Van HiepDHDuong Van Hiep
- Tran Cong TrinhTTTran Cong Trinh
- Ho Ngoc TrungHTHo Ngoc Trung
- Duong Quang TriDTDuong Quang Tri
Open Access
- 23 February 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 209-214
Abstract
Grasp generation plays a fundamental role in robot manipulation, often relying on three-dimensional (3D) point cloud data acquired through specialized depth cameras. However, the limited availability of such sensors in practical scenarios emphasizes the necessity for alternative approaches. This paper introduces an innovative method for grasp generation directly from color (RGB) images, negating the reliance on dedicated depth sensors. The proposed method employs tailored deep learning techniques for depth estimation from color images. Instead of traditional depth sensors, our approach computes predicted point clouds from estimated depth images directly generated from RGB inputs. A significant contribution lies in the design of a fusion module adept at seamlessly integrating features extracted from RGB images with those inferred from the predicted point clouds. This fusion process significantly strengthens the grasp generation pipeline by strengthening the advantages of both modalities, yielding notably improved grasp configurations. Experimental evaluations on standard datasets validate the efficacy of our approach, demonstrating its superior performance in generating grasp configurations compared to existing methods.Keywords
This publication has 37 references indexed in Scilit:
- Occlusion-Robust Pallet Pose Estimation for Warehouse AutomationIEEE Access, 2024
- Grasp Configuration Synthesis from 3D Point Clouds with Attention MechanismJournal of Intelligent & Robotic Systems, 2023
- Object-RPE: Dense 3D reconstruction and pose estimation with convolutional neural networksRobotics and Autonomous Systems, 2020
- Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a reviewArtificial Intelligence Review, 2020
- Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation2019
- Look Deeper into Depth: Monocular Depth Estimation with Semantic Booster and Attention-Driven Loss2018
- Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation2017
- Sub-OBB based object recognition and localization algorithm using range imagesMeasurement Science and Technology, 2016
- Deeper Depth Prediction with Fully Convolutional Residual Networks2016
- Deep Residual Learning for Image Recognition2016