A New CNN-RNN Framework For Remote Sensing Image Captioning
Authors:
- 1 March 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Abstract
Remote sensing (RS) image captioning has been recently attracting the attention of the community as it provides more semantic information with respect to the traditional tasks such as scene classification. Image captioning aims to generate a coherent and comprehensive description that summarizes the content of an image. The description can be obtained directly from the ground truth descriptions of similar images (retrieval based image captioning) or can be generated through the encoder-decoder framework. The former has the limitation of not generating new descriptions. The latter may be affected by misrecognition of scenes or semantic objects. In this paper we try to address these issues by proposing a new framework which is a combination of generation and retrieval based image captioning. First a CNN-RNN framework combined with beam-search generates multiple captions for a target image. Then the best caption is selected on the basis of its lexical similarity with the reference captions of most similar images. Experimental results on RSCID dataset are reported and discussed.Keywords
This publication has 7 references indexed in Scilit:
- Retrieving Images with Generated Textual Descriptions2019
- Semantic Descriptions of High-Resolution Remote Sensing ImagesIEEE Geoscience and Remote Sensing Letters, 2019
- Exploring Models and Data for Remote Sensing Image Caption GenerationIEEE Transactions on Geoscience and Remote Sensing, 2017
- Natural language description of remote sensing images based on deep learning2017
- Can a Machine Generate Humanlike Language Descriptions for a Remote Sensing Image?IEEE Transactions on Geoscience and Remote Sensing, 2017
- Rethinking the Inception Architecture for Computer Vision2016
- On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014