PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction
Authors:
Preprint
Abstract
We introduce a novel RGB-D patch descriptor designed for detecting coplanar surfaces in SLAM reconstruction. The core of our method is a deep convolutional neural net that takes in RGB, depth, and normal information of a planar patch in an image and outputs a descriptor that can be used to find coplanar patches from other images.We train the network on 10 million triplets of coplanar and non-coplanar patches, and evaluate on a new coplanarity benchmark created from commodity RGB-D scans. Experiments show that our learned descriptor outperforms alternatives extended for this new task by a significant margin. In addition, we demonstrate the benefits of coplanarity matching in a robust RGBD reconstruction formulation.We find that coplanarity constraints detected with our method are sufficient to get reconstruction results comparable to state-of-the-art frameworks on most scenes, but outperform other methods on standard benchmarks when combined with a simple keypoint method.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Focal Loss for Dense Object Detection2017
- Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras2017
- Joint Layout Estimation and Global Multi-view Registration for Indoor Reconstruction2017
- Probabilistic Combination of Noisy Points and Planes for RGB-D OdometryLecture Notes in Computer Science, 2017
- DSAC — Differentiable RANSAC for Camera Localization2017
- Fine-to-Coarse Global Registration of RGB-D Scans2017
- ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes2017
- Self-Supervised Visual Descriptor Learning for Dense CorrespondenceIEEE Robotics and Automation Letters, 2016
- Point-plane SLAM for hand-held 3D sensors2013
- Planar Features for Visual SLAMLecture Notes in Computer Science, 2008