Dynamic Scene Deblurring by Depth Guided Model
- 20 March 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 29 (10577149) , 5273-5288
Abstract
Dynamic scene blur is usually caused by object motion, depth variation as well as camera shake. Most existing methods usually solve this problem using image segmentation or fully end-to-end trainable deep convolutional neural networks by considering different object motions or camera shakes. However, these algorithms are less effective when there exist depth variations. In this work, we propose a deep neural convolutional network that exploits the depth map for dynamic scene deblurring. Given a blurred image, we first extract the depth map and adopt a depth refinement network to restore the edges and structure in the depth map. To effectively exploit the depth map, we adopt the spatial feature transform layer to extract depth features and fuse with the image features through scaling and shifting. Our image deblurring network thus learns to restore a clear image under the guidance of the depth map. With substantial experiments and analysis, we show that the depth information is crucial to the performance of the proposed model. Finally, extensive quantitative and qualitative evaluations demonstrate that the proposed model performs favorably against the state-of-the-art dynamic scene deblurring approaches as well as conventional depth-based deblurring algorithms.Keywords
Funding Information
- National Natural Science Foundation of China (61433007, 61872421, 61922043)
- Natural Science Foundation of Jiangsu Province (BK20180471)
- National Science Foundation (1149783)
This publication has 53 references indexed in Scilit:
- EpicFlow: Edge-preserving interpolation of correspondences for optical flow2015
- Generalized video deblurring for dynamic scenes2015
- Image Super-Resolution Using Deep Convolutional NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Segmentation-Free Dynamic Scene Deblurring2014
- Joint Depth Estimation and Camera Shake Removal from Single Blurry Image2014
- Unnatural L0 Sparse Representation for Natural Image Deblurring2013
- Interactive Facial Feature Localization2012
- Non-uniform Deblurring for Shaken ImagesInternational Journal of Computer Vision, 2011
- Two-Phase Kernel Estimation for Robust Motion Deblurring2010
- Removing camera shake from a single photograph2006