Single-View and Multi-View Depth Fusion

Authors:
Publisher website
Google Scholar
Abstract
Dense and accurate 3-D mapping from a monocular sequence is a key technology for several applications and still an open research area. This letter leverages recent results on single-view convolutional network (CNN)-based depth estimation and fuses them with multiview depth estimation. Both approaches present complementary strengths. Multiview depth is highly accurate but only in high-texture areas and high-parallax cases. Single-view depth captures the local structure of midlevel regions, including texture-less areas, but the estimated depth lacks global coherence. The single and multiview fusion we propose is challenging in several aspects. First, both depths are related by a deformation that depends on the image content. Second, the selection of multiview points of high accuracy might be difficult for low-parallax configurations. We present contributions for both problems. Our results in the public datasets of NYUv2 and TUM shows that our algorithm outperforms the individual single and multiview approaches. A video showing the key aspects of mapping in our single and multiview depth proposal is available at https://youtu.be/ipc5HukTb4k.
Funding Information
  • NVIDIA Corporation
  • Spanish Government (DPI2012-32168, DPI2015-67275)
  • Aragón Regional Government (DGA T04-FSE)
  • University of Zaragoza (JIUZ-2015-TEC-03)

This publication has 18 references indexed in Scilit: