X-Section: Cross-section Prediction for Enhanced RGBD Fusion
Authors:
Preprint
Abstract
Detailed 3D reconstruction is an important challenge with application to robotics, augmented and virtual reality, which has seen impressive progress throughout the past years. Advancements were driven by the availability of depth cameras (RGB-D), as well as increased compute power, e.g.\ in the form of GPUs -- but also thanks to inclusion of machine learning in the process. Here, we propose X-Section, an RGB-D 3D reconstruction approach that leverages deep learning to make object-level predictions about thicknesses that can be readily integrated into a volumetric multi-view fusion process, where we propose an extension to the popular KinectFusion approach. In essence, our method allows to complete shape in general indoor scenes behind what is sensed by the RGB-D camera, which may be crucial e.g.\ for robotic manipulation tasks or efficient scene exploration. Predicting object thicknesses rather than volumes allows us to work with comparably high spatial resolution without exploding memory and training data requirements on the employed Convolutional Neural Networks. In a series of qualitative and quantitative evaluations, we demonstrate how we accurately predict object thickness and reconstruct general 3D scenes containing multiple objects.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes2018
- ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans2018
- MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features2018
- DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image2018
- Efficient Octree-Based Volumetric SLAM Supporting Signed-Distance and Occupancy MappingIEEE Robotics and Automation Letters, 2018
- DCAN: Deep contour-aware networks for object instance segmentation from histology imagesMedical Image Analysis, 2017
- Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model SetIEEE Robotics & Automation Magazine, 2015
- Automatic photo pop-upACM Transactions on Graphics, 2005
- Single‐view modelling of free‐form scenesThe Journal of Visualization and Computer Animation, 2002
- Single View MetrologyInternational Journal of Computer Vision, 2000