Seeing Motion at Nighttime with an Event Camera
Preprint
Abstract
We focus on a very challenging task: imaging at nighttime dynamic scenes. Most previous methods rely on the low-light enhancement of a conventional RGB camera. However, they would inevitably face a dilemma between the long exposure time of nighttime and the motion blur of dynamic scenes. Event cameras react to dynamic changes with higher temporal resolution (microsecond) and higher dynamic range (120dB), offering an alternative solution. In this work, we present a novel nighttime dynamic imaging method with an event camera. Specifically, we discover that the event at nighttime exhibits temporal trailing characteristics and spatial non-stationary distribution. Consequently, we propose a nighttime event reconstruction network (NER-Net) which mainly includes a learnable event timestamps calibration module (LETC) to align the temporal trailing events and a non-uniform illumination aware module (NIAM) to stabilize the spatiotemporal distribution of events. Moreover, we construct a paired real low-light event dataset (RLED) through a co-axial imaging system, including 64,200 spatially and temporally aligned image GTs and low-light events. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods in terms of visual quality and generalization ability on real-world nighttime datasets. The project are available at: https://github.com/Liu-haoyue/NER-Net.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- DSEC: A Stereo Event Camera Dataset for Driving ScenariosIEEE Robotics and Automation Letters, 2021
- Beyond Brightening Low-light ImagesInternational Journal of Computer Vision, 2021
- A 1280×960 Dynamic Vision Sensor with a 4.95-μm Pixel Pitch and Motion Artifact Minimization2020
- Hybrid enhancement of infrared night vision imaging systemMultimedia Tools and Applications, 2019
- Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold RegularisationInternational Journal of Computer Vision, 2018
- Live Demonstration: Front and Back Illuminated Dynamic and Active Pixel Vision Sensors Comparison2018
- LLNet: A deep autoencoder approach to natural low-light image enhancementPattern Recognition, 2017
- A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDSIEEE Journal of Solid-State Circuits, 2010
- Properties and performance of a center/surround retinexIEEE Transactions on Image Processing, 1997
- Visual adaptation in relation to brief conditioning stimuliProceedings of the Royal Society of London. B. Biological Sciences, 1947