Event-based Motion Deblurring with Modality-Aware Decomposition and Recomposition
Open Access
- 26 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 8327-8335
Abstract
Event camera responds to the brightness changes at each pixel independently with microsecond accuracy. Event cameras offer attractive property that can record well high-speed scene but ignore static and non-moving areas, while conventional frame cameras are able to acquire the whole intensity information of the scene but suffer from motion blur. Therefore, it would be desirable to combine the best of two cameras for reconstructing high quality intensity frame with no motion blur. The human visual system presents a two-pathway procedure for non-action-based representation and objects motion perception, which corresponds well to the hybrid frame and event. In this paper, inspired by the two-pathway visual system, a novel dual-stream based framework is proposed for motion deblurring (DS-Deblur), which flexibly utilizes the respective advantages from frame and event. A complementary-unique information splitting based feature fusion module is firstly proposed to adaptively aggregate the frame and event progressively at multiple levels, which is well-grounded on the hierarchical process in twopathway visual system. Then, a recurrent spatio-temporal feature transformation module is designed to exploit relevant information between adjacent frames, in which features of both current and previous frames are transformed in a global-local manner. Extensive experiments on both synthetic and real motion blur datasets demonstrate our method achieves state-of-the-art performance. Project website: https://github.com/wyang-vis/Motion-Deblurringwith-Hybrid-Frames-and-Events.Keywords
Funding Information
- National Natural Science Foundation of China (62022063)
This publication has 27 references indexed in Scilit:
- Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring2019
- Continuous-Time Intensity Estimation Using Event Cameras2019
- Image Super-Resolution Using Very Deep Residual Channel Attention Networks2018
- Scale-Recurrent Network for Deep Image Deblurring2018
- Non-uniform Blind Deblurring by Reblurring2017
- Modeling Blurred Video with Layers2014
- Blind deconvolution using a normalized sparsity measure2011
- Motion from blur2008
- A Variational Framework for Simultaneous Motion Estimation and Restoration of Motion-Blurred Video2007
- Removing camera shake from a single photograph2006