NFL: Normal Field Learning for 6-DoF Grasping of Transparent Objects
- 23 November 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (1) , 819-826
Abstract
We present Normal Field Learning (NFL), a robust yet practical solution to perceive 3D layouts of transparent objects and grasp them quickly. Conventional input modalities for vision-based grasping do not provide sufficient information for transparent objects. However, with the recent advance on datasets and algorithms for transparent objects, we can at least obtain noisy estimates of normals and masks for various real-world conditions. Instead of directly using the RGB images, we propose to use the estimates to train a neural volume, which serves as an intermediate representation ignorant of challenging appearance variations. We formulate the training objective to account for inherent uncertainty in individual estimation, and together with the volumetric aggregation, we can reliably extract useful geometric information for grasping. Our neural volume deploys a voxel-grid based representation, motivated by acceleration techniques of neural radiance fields. However, we directly store the normal and density values in the grid cells instead of latent features. Our modification allows direct access to the geometric values without additional inference or volume rendering, further enhancing the efficiency. Our results show over 85% success rates in grasping in cluttered scenes with only 40 seconds of training time.Keywords
Funding Information
- CAINS (KIAS Individual)
- Natural Sciences at Korea Institute (AP092701)
- National Research Foundation of Korea
- Ministry of Science and ICT, South Korea (RS-2023-00208197)
- Institute of Information & Communications Technology Planning & Evaluation (2021-0-01343)
- Artificial Intelligence Graduate School Program
- Seoul National University
This publication has 32 references indexed in Scilit:
- Segmenting Transparent Objects in the Wild2020
- Don’t Hit Me! Glass Detection in Real-World Scenes2020
- Deep Polarization Cues for Transparent Object Segmentation2020
- Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation2020
- TOM-Net: Learning Transparent Object Matting from a Single Image2018
- Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics2017
- GP-GPIS-OPT: Grasp planning with shape uncertainty using Gaussian process implicit surfaces and Sequential Convex Programming2015
- Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengthsJournal of the Optical Society of America A, 2002
- Statistical Analysis of Spherical Data1987
- Ray tracing volume densitiesACM SIGGRAPH Computer Graphics, 1984