Fine Tuning of Deep Contexts Toward Improved Perceptual Quality of In-Paintings
- 8 September 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 52 (6) , 4850-4854
Abstract
Over the recent years, a number of deep learning approaches are successfully introduced to tackle the problem of image in-painting for achieving better perceptual effects. However, there still exist obvious hole-edge artifacts in these deep learning-based approaches, which need to be rectified before they become useful for practical applications. In this article, we propose an iteration-driven in-painting approach, which combines the deep context model with the backpropagation mechanism to fine-tune the learning-based in-painting process and hence, achieves further improvement over the existing state of the arts. Our iterative approach fine tunes the image generated by a pretrained deep context model via backpropagation using a weighted context loss. Extensive experiments on public available test sets, including the CelebA, Paris Streets, and PASCAL VOC 2012 dataset, show that our proposed method achieves better visual perceptual quality in terms of hole-edge artifacts compared with the state-of-the-art in-painting methods using various context models.Keywords
Funding Information
- National Natural Science Foundation of China (61602312, 61620106008)
- Shenzhen Emerging Industries of the Strategic Basic Research Project (JCYJ20160226191842793)
- natural science foundation China (61620106008, 62032015)
This publication has 21 references indexed in Scilit:
- Deep Learning Face Attributes in the Wild2015
- Statistics of Patch Offsets for Image Completion2012
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- PatchMatchACM Transactions on Graphics, 2009
- Get Out of my Picture! Internet-based Inpainting2009
- Scene completion using millions of photographsCommunications of the ACM, 2008
- Texture optimization for example-based synthesisACM Transactions on Graphics, 2005
- Fragment-based image completion2003
- Graphcut texturesACM Transactions on Graphics, 2003
- Image quilting for texture synthesis and transfer2001