EDFLOW: Event Driven Optical Flow Camera With Keypoint Detection and Adaptive Block Matching
Top Cited Papers
- 3 March 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 32 (9) , 5776-5789
Abstract
Event cameras such as the Dynamic Vision Sensor (DVS) are useful because of their low latency, sparse output, and high dynamic range. In this paper, we propose a DVS+FPGA camera platform and use it to demonstrate the hardware implementation of event-based corner keypoint detection and adaptive block-matching optical flow. To adapt sample rate dynamically, events are accumulated in event slices using the area event count slice exposure method. The area event count is feedback controlled by the average optical flow matching distance. Corners are detected by streaks of accumulated events on event slice rings of radius 3 and 4 pixels. Corner detection takes about 6 clock cycles (16 MHz event rate at the 100MHz clock frequency) At the corners, flow vectors are computed in 100 clock cycles (1 MHz event rate). The multiscale block match size is $25\times 25$ pixels and the flow vectors span up to 30-pixel match distance. The FPGA processes the sum-of-absolute distance block matching at 123 GOp/s, the equivalent of 1230 Op/clock cycle. EDFLOW is several times more accurate on MVSEC drone and driving optical flow benchmarking sequences than the previous best DVS FPGA optical flow implementation, and achieves similar accuracy to the CNN-based EV-Flownet, although it burns about 100 times less power. The EDFLOW design and benchmarking videos are available at https://sites.google.com/view/edflow21/home .Keywords
Funding Information
- Swiss National Centre of Competence in Research Robotics
This publication has 58 references indexed in Scilit:
- A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision SensorIEEE Journal of Solid-State Circuits, 2014
- Optic-Flow Based Slope Estimation for Autonomous LandingInternational Journal of Micro Air Vehicles, 2013
- Event-Based Visual FlowIEEE Transactions on Neural Networks and Learning Systems, 2013
- Asynchronous frameless event-based optical flowNeural Networks, 2012
- A Database and Evaluation Methodology for Optical FlowInternational Journal of Computer Vision, 2010
- Vision-based control of near-obstacle flightAutonomous Robots, 2009
- A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision SensorIEEE Journal of Solid-State Circuits, 2008
- A neuromorphic aVLSI model of global motion processing in the flyIEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 2000
- Silicon retina with correlation-based, velocity-tuned pixelsIEEE Transactions on Neural Networks, 1993
- Array architectures for block matching algorithmsIEEE Transactions on Circuits and Systems, 1989