Higher Order Function Networks for View Planning and Multi-View Reconstruction
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2588, 11486-11492
Abstract
We consider the problem of planning views for a robot to acquire images of an object for visual inspection and reconstruction. In contrast to offline methods which require a 3D model of the object as input or online methods which rely on only local measurements, our method uses a neural network which encodes shape information for a large number of objects. We build on recent deep learning methods capable of generating a complete 3D reconstruction of an object from a single image. Specifically, in this work, we extend a recent method which uses Higher Order Functions (HOF) to represent the shape of the object. We present a new generalization of this method to incorporate multiple images as input and establish a connection between visibility and reconstruction quality. This relationship forms the foundation of our view planning method where we compute viewpoints to visually cover the output of the multiview HOF network with as few images as possible. Experiments indicate that our method provides a good compromise between online and offline methods: Similar to online methods, our method does not require the true object model as input. In terms of number of views, it is much more efficient. In most cases, its performance is comparable to the optimal offline case even on object classes the network has not been trained on.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Sampling-based view planning for 3D visual coverage task with Unmanned Aerial Vehicle2016
- 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction2016
- Mutual information-based exploration on continuous occupancy maps2015
- DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time2015
- On mutual information-based control of range sensing robots for mapping applicationsThe International Journal of Robotics Research, 2014
- View planning for automated three-dimensional object reconstruction and inspectionACM Computing Surveys, 2003
- Navigation Strategies for Exploring Indoor EnvironmentsThe International Journal of Robotics Research, 2002
- Planning for Complete Sensor Coverage in InspectionComputer Vision and Image Understanding, 1995
- Occlusions as a guide for planning the next viewIEEE Transactions on Pattern Analysis and Machine Intelligence, 1993
- ɛ-nets and simplex range queriesDiscrete & Computational Geometry, 1987