Vehicle Re-identification: exploring feature fusion using multi-stream convolutional networks
Authors:
- Icaro O. de OliveiraIOIcaro O. de Oliveira
- Rayson LarocaRLRayson Laroca
- David MenottiDMDavid Menotti
- Keiko V. O. FonsecaKFKeiko V. O. Fonseca
- Rodrigo MinettoRMRodrigo Minetto
Preprint
Abstract
This work addresses the problem of vehicle re-identification through a network of non-overlapping cameras. As our main contribution, we propose a novel two-stream convolutional neural network (CNN) that simultaneously uses two of the most distinctive and persistent features available: the vehicle appearance and its license plate. This is an attempt to tackle a major problem, false alarms caused by vehicles with similar design or by very close license plate identifiers. In the first network stream, shape similarities are identified by a Siamese CNN that uses a pair of low-resolution vehicle patches recorded by two different cameras. In the second stream, we use a CNN for optical character recognition (OCR) to extract textual information, confidence scores, and string similarities from a pair of high-resolution license plate patches. Then, features from both streams are merged by a sequence of fully connected layers for decision. As part of this work, we created an important dataset for vehicle re-identification with more than three hours of videos spanning almost 3,000 vehicles. In our experiments, we achieved a precision, recall and F -score values of 99.6%, 99.2% and 99.4%, respectively. As another contribution, we discuss and compare three alternative architectures that explore the same features but using additional streams and temporal information. The proposed architectures, trained models, and dataset are publicly available at https://github.com/icarofua/vehicle-ReId .Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- An efficient and layout‐independent automatic license plate recognition system based on the YOLO detectorIET Intelligent Transport Systems, 2021
- Embedding Adversarial Learning for Vehicle Re-IdentificationIEEE Transactions on Image Processing, 2019
- Supervised Deep Feature Embedding With Handcrafted FeatureIEEE Transactions on Image Processing, 2019
- Reading car license plates using deep neural networksImage and Vision Computing, 2018
- A Two Stream Siamese Convolutional Neural Network for Person Re-identification2017
- Learning Contextual Dependence With Convolutional Hierarchical Recurrent Neural NetworksIEEE Transactions on Image Processing, 2016
- A survey of approaches and trends in person re-identificationImage and Vision Computing, 2014
- T-HOG: An effective gradient-based descriptor for single line text regionsPattern Recognition, 2013
- Efficient multi-camera vehicle detection, tracking, and identification in a tunnel surveillance applicationComputer Vision and Image Understanding, 2012
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004