Context R-CNN: Long Term Temporal Context for Per-Camera Object Detection
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13072-13082
Abstract
In static monitoring cameras, useful contextual information can stretch far beyond the few seconds typical video understanding models might see: subjects may exhibit similar behavior over multiple days, and background objects remain static. Due to power and storage constraints, sampling frequencies are low, often no faster than one frame per second, and sometimes are irregular due to the use of a motion trigger. In order to perform well in this setting, models must be robust to irregular sampling rates. In this paper we propose a method that leverages temporal context from the unlabeled frames of a novel camera to improve performance at that camera. Specifically, we propose an attention-based approach that allows our model, Context R-CNN, to index into a long term memory bank constructed on a per-camera basis and aggregate contextual features from other frames to boost object detection performance on the current frame. We apply Context R-CNN to two settings: (1) species detection using camera traps, and (2) vehicle detection in traffic cameras, showing in both settings that Context R-CNN leads to performance gains over strong baselines. Moreover, we show that increasing the contextual time horizon leads to improved results. When applied to camera trap data from the Snapshot Serengeti dataset, Context R-CNN with context from up to a month of images outperforms a single-frame baseline by 17.9% mAP, and outperforms S3D (a 3d convolution based baseline) by 11.2% mAP.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- Towards automatic wild animal monitoring: Identification of animal species in camera-trap images using very deep convolutional neural networksEcological Informatics, 2017
- Deep Feature Flow for Video Recognition2017
- Fast human-animal detection from highly cluttered camera-trap images using joint background modeling and deep learning classification2017
- Finding areas of motion in camera trap images2016
- Animal Detection From Highly Cluttered Natural Scenes Using Spatiotemporal Object Region Proposals and Patch VerificationIEEE Transactions on Multimedia, 2016
- Deep Residual Learning for Image Recognition2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savannaScientific Data, 2015
- Finding action tubes2015
- Automated identification of animal species in camera trap imagesEURASIP Journal on Image and Video Processing, 2013