Video Object Detection With Two-Path Convolutional LSTM Pyramid
Open Access
- 17 August 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 151681-151691
Abstract
One of the major challenges in video object detection is drastic scale changes of objects due to camera motion. In this paper, we propose a two-path Convolutional Long Short-Term Memory (convLSTM) pyramid network designed to extract and convey multi-scale temporal contextual information in order to handle object scale changes efficiently. The proposed two-path convLSTM pyramid consists of a stack of multi-input convLSTM modules. It is updated in top-down and bottom-up pathways so that the temporal contextual information for small-to-large and large-to-small scale changes is exploited. The proposed multi-input convLSTM module uses two input feature maps of different resolutions to store and exchange temporal contextual information of different scales between neighboring convLSTM modules. The outputs of the proposed convLSTM pyramid network constitute a feature pyramid where each feature map contains multi-scale temporal contextual information from earlier frames. The proposed convLSTM pyramid can be combined with various still-image object detectors to improve the performance of video object detection. Experimental results on ImageNet VID dataset show that the proposed method achieves state-of-the-art performance and can handle scale changes efficiently in video object detection.Keywords
Funding Information
- Industrial Core Technology Development Program of MOTIEKEIT KOREA (10083639)
This publication has 36 references indexed in Scilit:
- FlowNet: Learning Optical Flow with Convolutional Networks2015
- Fast R-CNN2015
- Region-Based Convolutional Networks for Accurate Object Detection and SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Speech recognition with deep recurrent neural networks2013
- Integral Channel Features2009
- Histograms of Oriented Gradients for Human Detection2005
- Long Short-Term MemoryNeural Computation, 1997
- Nonlinear dynamic system identification using artificial neural networks (ANNs)1990
- A massively parallel architecture for a self-organizing neural pattern recognition machineComputer Vision, Graphics, and Image Processing, 1986