A news picture geo-localization pipeline based on deep learning and street view images
Authors:
Open Access
- 12 September 2022
- journal article
- research article
- Published by Taylor & Francis in International Journal of Digital Earth
- Vol. 15 (1) , 1485-1505
Abstract
Numerous news or event pictures are taken and shared on the internet every day that have abundant information worth being mined, but only a small fraction of them are geotagged. The visual content of the news image hints at clues of the geographical location because they are usually taken at the site of the incident, which provides a prerequisite for geo-localization. This paper proposes an automated pipeline based on deep learning for the geo-localization of news pictures in a large-scale urban environment using geotagged street view images as a reference dataset. The approach obtains location information by constructing an attention-based feature extraction network. Then, the image features are aggregated, and the candidate street view image results are retrieved by the selective matching kernel function. Finally, the coordinates of the news images are estimated by the kernel density prediction method. The pipeline is tested in the news pictures in Hong Kong. In the comparison experiments, the proposed pipeline shows stable performance and generalizability in the large-scale urban environment. In addition, the performance analysis of components in the pipeline shows the ability to recognize localization features of partial areas in pictures and the effectiveness of the proposed solution in news picture geo-localization.Keywords
This publication has 57 references indexed in Scilit:
- Cross-Dimensional Weighting for Aggregated Deep Convolutional Features2016
- Aggregating Local Deep Features for Image Retrieval2015
- Image Search with Selective Match Kernels: Aggregation Across Single and Multiple ImagesInternational Journal of Computer Vision, 2015
- Visual Place Recognition with Repetitive StructuresIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Localization of mobile laser scanner using classical mechanicsISPRS Journal of Photogrammetry and Remote Sensing, 2015
- All About VLAD2013
- Using Google Street View to Audit the Built Environment: Inter-rater Reliability ResultsAnnals of Behavioral Medicine, 2012
- Automatic Caption Generation for News ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Fisher Kernels on Visual Vocabularies for Image Categorization2007
- Video Google: a text retrieval approach to object matching in videos2003