Attention boosted bilinear pooling for remote sensing image retrieval
- 8 December 2019
- journal article
- research article
- Published by Taylor & Francis in International Journal of Remote Sensing
- Vol. 41 (7) , 2704-2724
Abstract
Remote sensing image retrieval is to find the most identical or similar images to a query image in the vast archive of remote sensing images. A key process is to extract the most distinctive features. In this study, we introduce a second-order pooling named compact bilinear pooling (CBP) into convolutional neural networks (CNNs) for remote sensing image retrieval. The retrieval algorithm has three stages, pretraining, fine-tuning and retrieval. In the pretraining stage, two classic CNN structures, VGG16 and ResNet34, are pretrained respectively with the ImageNet consisting of close-range images. A CBP layer is introduced before the fully connected layers in the two networks. To extract globally consistent representations, a channel and spatial integrated attention mechanism is proposed to refine features from the last convolution layer and the features are used as the input of the CBP. In the fine-tuning stage, the new network is fine-tuned on a remote sensing dataset to train discriminable features. In the retrieval stage, the network, with fully connected layers being replaced by a PCA (principal component analysis) module, is applied to new remote sensing datasets. Our retrieval algorithm with the combination of CBP and PCA obtained the best performance and outperformed several mainstream pooling or encoding methods such as full-connected layer, IFK (Improved Fisher Kernel), BoW (Bag-of-Words) and maxpooling, etc. The channel and spatial attention mechanism contributes to the CBP based retrieval method and obtained the best performance on all the datasets, as well as outperformed several recent attention methods. Source code is available at http://study.rsgis/whu.edu.cn/pages/download.Keywords
This publication has 16 references indexed in Scilit:
- Deep Metric Learning Using Triplet Network2015
- Deep Learning Based Feature Selection for Remote Sensing Scene ClassificationIEEE Geoscience and Remote Sensing Letters, 2015
- Deep learningNature, 2015
- Dynamic Texture Recognition Using Volume Local Binary Patterns2007
- Integrated color, texture and shape information for content-based image retrievalPattern Analysis and Applications, 2007
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Multiresolution gray-scale and rotation invariant texture classification with local binary patternsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Color and Shape Index for Region-Based Image Retrieval2001
- Nonlinear Dimensionality Reduction by Locally Linear EmbeddingScience, 2000
- Texture feature performance for image segmentationPattern Recognition, 1990