Object Guided External Memory Network for Video Object Detection
Authors:
- 1 October 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6677-6686
Abstract
Video object detection is more challenging than image object detection because of the deteriorated frame quality. To enhance the feature representation, state-of-the-art methods propagate temporal information into the deteriorated frame by aligning and aggregating entire feature maps from multiple nearby frames. However, restricted by feature map's low storage-efficiency and vulnerable content-address allocation, long-term temporal information is not fully stressed by these methods. In this work, we propose the first object guided external memory network for online video object detection. Storage-efficiency is handled by object guided hard-attention to selectively store valuable features, and long-term information is protected when stored in an addressable external data matrix. A set of read/write operations are designed to accurately propagate/allocate and delete multi-level memory feature under object guidance. We evaluate our method on the ImageNet VID dataset and achieve state-of-the-art performance as well as good speed-accuracy tradeoff. Furthermore, by visualizing the external memory, we show the detailed object-level reasoning process across frames.Keywords
This publication has 21 references indexed in Scilit:
- Hybrid computing using a neural network with dynamic external memoryNature, 2016
- Object Detection from Video Tubelets with Convolutional Neural Networks2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- FlowNet: Learning Optical Flow with Convolutional Networks2015
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Fast R-CNN2015
- Going deeper with convolutions2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Long Short-Term MemoryNeural Computation, 1997