PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3711-3721
Abstract
The rising availability of commercial 360° cameras that democratize indoor scanning, has increased the interest for novel applications, such as interior space re-design. Diminished Reality (DR) fulfills the requirement of such applications, to remove existing objects in the scene, essentially translating this to a counterfactual inpainting task. While recent advances in data-driven inpainting have shown significant progress in generating realistic samples, they are not constrained to produce results with reality mapped structures. To preserve the ‘reality’ in indoor (re-)planning applications, the scene’s structure preservation is crucial. To ensure structure-aware counterfactual inpainting, we propose a model that initially predicts the structure of a indoor scene and then uses it to guide the reconstruction of an empty – background only – representation of the same scene. We train and compare against other state-of-the-art methods on a version of the Structured3D dataset [47] modified for DR, showing superior results in both quantitative metrics and qualitative results, but more interestingly, our approach exhibits a much faster convergence rate. Code and models are available at github.com/VCL3D/PanoDR/Keywords
This publication has 33 references indexed in Scilit:
- Photographic Image Synthesis with Cascaded Refinement Networks2017
- Globally and locally consistent image completionACM Transactions on Graphics, 2017
- Dilated Residual Networks2017
- Image Style Transfer Using Convolutional Neural Networks2016
- View Synthesis by Appearance Flow2016
- Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks2016
- Omnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Structure extraction from texture via relative total variationACM Transactions on Graphics, 2012
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004