EDBGAN: Image Inpainting via an Edge-Aware Dual Branch Generative Adversarial Network
- 6 April 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 28 (10709908) , 842-846
Abstract
As deep learning technology develops rapidly, image inpainting methods have made significant progress in generating reasonable contents for images with large and irregular holes. Nevertheless, existing methods either use one encoder-decoder to generate results in one step without making good use of structure features which are helpful, or adopt two encoder-decoders to recover structures and textures subsequently where the second encoder-decoder used for recovering textures relies heavily on the first encoder-decoder. Thus, an one-stage method which simultaneously utilizes structures and textures is promising. In this letter, we propose a dual branch encoder-decoder, whose texture branch and edge branch can simultaneously extract features from the masked RGB image and its corresponding edge map. Moreover, we propose a lightweight mutually guided attention block (MGAB), which makes features of two branches guide each other from low level to high level. At the end of two branches, we apply dual attention block (DAB) to perform feature fusion, which uses self-attention mechanism and lightweight attention mechanism on features of texture branch and edge branch, respectively. Extensive experiments demonstrate that our proposed method is effective in recovering edges and textures and achieves the state-of-the-art performance.Keywords
Funding Information
- National Natural Science Foundation of China (61771301)
This publication has 22 references indexed in Scilit:
- Shift-Net: Image Inpainting via Deep Feature Rearrangement2018
- Globally and locally consistent image completionACM Transactions on Graphics, 2017
- Places: A 10 Million Image Database for Scene RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis2017
- View Synthesis by Appearance Flow2016
- Context Encoders: Feature Learning by Inpainting2016
- Rethinking the Inception Architecture for Computer Vision2016
- Deep Learning Face Attributes in the Wild2015
- Image Quality Metrics: PSNR vs. SSIM2010
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004