Learning-Based Sampling for Natural Image Matting
Authors:
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3050-3058
Abstract
The goal of natural image matting is the estimation of opacities of a user-defined foreground object that is essential in creating realistic composite imagery. Natural matting is a challenging process due to the high number of unknowns in the mathematical modeling of the problem, namely the opacities as well as the foreground and background layer colors, while the original image serves as the single observation. In this paper, we propose the estimation of the layer colors through the use of deep neural networks prior to the opacity estimation. The layer color estimation is a better match for the capabilities of neural networks, and the availability of these colors substantially increase the performance of opacity estimation due to the reduced number of unknowns in the compositing equation. A prominent approach to matting in parallel to ours is called sampling-based matting, which involves gathering color samples from known-opacity regions to predict the layer colors. Our approach outperforms not only the previous hand-crafted sampling algorithms, but also current data-driven methods. We hence classify our method as a hybrid sampling- and learning-based approach to matting, and demonstrate the effectiveness of our approach through detailed ablation studies using alternative network architectures.Keywords
This publication has 16 references indexed in Scilit:
- Globally and locally consistent image completionACM Transactions on Graphics, 2017
- Places: A 10 Million Image Database for Scene RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Image Matting with KL-Divergence Based Sparse Sampling2015
- Mask-Specific Inpainting with Deep Neural Networks2014
- Weighted Color and Texture Sample Selection for Image MattingIEEE Transactions on Image Processing, 2013
- KNN MattingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Manifold preserving edit propagationACM Transactions on Graphics, 2012
- Shared Sampling for Real‐Time Alpha MattingComputer Graphics Forum, 2010
- A Closed-Form Solution to Natural Image MattingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- Optimized Color Sampling for Robust Matting2007