FlexPatch: Fast and Accurate Object Detection for On-device High-Resolution Live Video Analytics
Authors:
- 2 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1898-1907
Abstract
We present FlexPatch, a novel mobile system to enable accurate and real-time object detection over high-resolution video streams. A widely-used approach for real-time video analysis is detection-based tracking (DBT), i.e., running the heavy-but-accurate detector every few frames and applying a lightweight tracker for in-between frames. However, the approach is limited for real-time processing of high-resolution videos in that i) a lightweight tracker fails to handle occlusion, object appearance changes, and occurrences of new objects, and ii) the detection results do not effectively offset tracking errors due to the high detection latency. We propose tracking-aware patching technique to address such limitations of the DBT frameworks. It effectively identifies a set of subareas where the tracker likely fails and tightly packs them into a small-sized rectangular area where the detection can be efficiently performed at low latency. This prevents the accumulation of tracking errors and offsets the tracking errors with frequent fresh detection results. Our extensive evaluation shows that FlexPatch not only enables real-time and power-efficient analysis of high-resolution frames on mobile devices but also improves the overall accuracy by 146% compared to baseline DBT frameworks.Keywords
Funding Information
- Samsung
This publication has 27 references indexed in Scilit:
- Focal Loss for Dense Object Detection2017
- Simple online and realtime tracking with a deep association metric2017
- Okutama-Action: An Aerial View Video Dataset for Concurrent Human Action Detection2017
- DeepMon2017
- Glimpse2015
- ORB: An efficient alternative to SIFT or SURF2011
- Real Time Pedestrian Tracking using Thermal Infrared ImageryJournal of Computers, 2010
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- A Computational Approach to Edge DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1986
- Computers and Intractability: A Guide to the Theory of NP-Completeness (Michael R. Garey and David S. Johnson)SIAM Review, 1982