Domain-Specific Image Caption Generator with Semantic Ontology
Authors:
- 1 February 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 526-530
Abstract
Image captioning is the task of generating textual descriptions of a given image, requiring techniques of computer vision and natural language processing. Recent models have utilized deep learning techniques for this task to gain performance improvement. However, these models can neither fully use information included in a given image such as object and attribute, nor generate a domain-specific caption because existing methods use open dataset such as MSCOCO which include general images. To overcome these limitations, this paper proposes a domain-specific image caption generator, which generates a caption based on attention mechanism with object and attribute information, and reconstruct a generate caption using a semantic ontology to provide natural language description for given specific-domain. To show the effectiveness of the proposed model, we evaluate the image caption generator with a dataset, MSCOCO, quantitatively and qualitatively.Keywords
This publication has 5 references indexed in Scilit:
- Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering2018
- Mask R-CNN2017
- Attention Correctness in Neural Image CaptioningProceedings of the AAAI Conference on Artificial Intelligence, 2017
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image AnnotationsInternational Journal of Computer Vision, 2017
- Deep visual-semantic alignments for generating image descriptions2015