FCN-rLSTM: Deep Spatio-Temporal Neural Networks for Vehicle Counting in City Cameras
Authors:
- Shanghang ZhangSZShanghang Zhang
- Guanhang WuGWGuanhang Wu
- Joao P. CosteiraJCJoao P. Costeira
- Jose M. F. MouraJMJose M. F. Moura
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3687-3696
Abstract
In this paper, we develop deep spatio-temporal neural networks to sequentially count vehicles from low quality videos captured by city cameras (citycams). Citycam videos have low resolution, low frame rate, high occlusion and large perspective, making most existing methods lose their efficacy. To overcome limitations of existing methods and incorporate the temporal information of traffic video, we design a novel FCN-rLSTM network to jointly estimate vehicle density and vehicle count by connecting fully convolutional neural networks (FCN) with long short term memory networks (LSTM) in a residual learning fashion. Such design leverages the strengths of FCN for pixel-level prediction and the strengths of LSTM for learning complex temporal dynamics. The residual learning connection reformulates the vehicle count regression as learning residual functions with reference to the sum of densities in each frame, which significantly accelerates the training of networks. To preserve feature map resolution, we propose a Hyper-Atrous combination to integrate atrous convolution in FCN and combine feature maps of different convolution layers. FCN-rLSTM enables refined feature representation and a novel end-to-end trainable mapping from pixels to vehicle count. We extensively evaluated the proposed method on different counting tasks with three datasets, with experimental results demonstrating their effectiveness and robustness. In particular, FCN-rLSTM reduces the mean absolute error (MAE) from 5.31 to 4.21 on TRANCOS; and reduces the MAE from 2.74 to 1.53 on WebCamT. Training process is accelerated by 5 times on average.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- Extremely Overlapping Vehicle Counting2015
- Smart traffic lights switching and traffic density calculation using video processing2014
- Cumulative Attribute Space for Age and Crowd Density Estimation2013
- Intelligent Moving Objects Detection via Adaptive Frame Differencing Method2013
- Feature Mining for Localised Crowd Counting2012
- Face Recognition Using Kernel Ridge Regression2007
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- A survey of video processing techniques for traffic applicationsImage and Vision Computing, 2003
- Statistic and knowledge-based moving object detection in traffic scenes2002
- Long Short-Term MemoryNeural Computation, 1997