Data-driven Feature Tracking for Event Cameras
Authors:
Preprint
Abstract
Because of their high temporal resolution, increased resilience to motion blur, and very sparse output, event cameras have been shown to be ideal for low-latency and low-bandwidth feature tracking, even in challenging scenarios. Existing feature tracking methods for event cameras are either handcrafted or derived from first principles but require extensive parameter tuning, are sensitive to noise, and do not generalize to different scenarios due to unmodeled effects. To tackle these deficiencies, we introduce the first data-driven feature tracker for event cameras, which leverages low-latency events to track features detected in a grayscale frame. We achieve robust performance via a novel frame attention module, which shares information across feature tracks. By directly transferring zero-shot from synthetic to real data, our data-driven tracker outperforms existing approaches in relative feature age by up to 120% while also achieving the lowest latency. This performance gap is further increased to 130% by adapting our tracker to real data with a novel self-supervision strategy.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Dense Continuous-Time Optical Flow From Event CamerasIEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
- Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories2022
- EDFLOW: Event Driven Optical Flow Camera With Keypoint Detection and Adaptive Block MatchingIEEE Transactions on Circuits and Systems for Video Technology, 2022
- Patch2Pix: Epipolar-Guided Pixel-Level Correspondences2021
- Event-Based Vision: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- EKLT: Asynchronous Photometric Feature Tracking Using Events and FramesInternational Journal of Computer Vision, 2019
- ACE: An Efficient Asynchronous Corner Tracker for Event Cameras2018
- Asynchronous Corner Detection and Tracking for Event Cameras in Real TimeIEEE Robotics and Automation Letters, 2018
- Event-Based, 6-DOF Camera Tracking from Photometric Depth MapsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Asynchronous Event-Based Visual Shape Tracking for Stable Haptic Feedback in MicroroboticsIEEE Transactions on Robotics, 2012