Deep Video Inpainting
Authors:
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5785-5794
Abstract
Video inpainting aims to fill spatio-temporal holes with plausible content in a video. Despite tremendous progress of deep neural networks for image inpainting, it is challenging to extend these methods to the video domain due to the additional time dimension. In this work, we propose a novel deep network architecture for fast video inpainting. Built upon an image-based encoder-decoder model, our framework is designed to collect and refine information from neighbor frames and synthesize still-unknown regions. At the same time, the output is enforced to be temporally consistent by a recurrent feedback and a temporal memory module. Compared with the state-of-the-art image inpainting algorithm, our method produces videos that are much more semantically correct and temporally smooth. In contrast to the prior video completion method which relies on time-consuming optimization, our method runs in near real-time while generating competitive video results. Finally, we applied our framework to video retargeting task, and obtain visually pleasing results.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Video Inpainting of Complex ScenesSIAM Journal on Imaging Sciences, 2014
- How Not to Be Seen — Object Removal from Videos of Crowded ScenesComputer Graphics Forum, 2012
- Background Inpainting for Videos with Dynamic Objects and a Free-Moving Camera2012
- PatchMatchACM Transactions on Graphics, 2009
- Video Inpainting Under Constrained Camera MotionIEEE Transactions on Image Processing, 2007
- Video Completion by Motion Field Transfer2006
- Video inpainting of occluding and occluded objects2005
- Space-time video completion2004
- Filling-in by joint interpolation of vector fields and gray levelsIEEE Transactions on Image Processing, 2001
- Image inpainting2000