Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation
Authors:
Top Cited Papers
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5402-5412
Abstract
Have you ever looked at a painting and wondered what is the story behind it? This work presents a framework to bring art closer to people by generating comprehensive descriptions of fine-art paintings. Generating informative descriptions for artworks, however, is extremely challenging, as it requires to 1) describe multiple aspects of the image such as its style, content, or composition, and 2) provide background and contextual knowledge about the artist, their influences, or the historical period. To address these challenges, we introduce a multi-topic and knowledgeable art description framework, which modules the generated sentences according to three artistic topics and, additionally, enhances each description with external knowledge. The framework is validated through an exhaustive analysis, both quantitative and qualitative, as well as a comparative human evaluation, demonstrating outstanding results in terms of both topic diversity and information veracity.Keywords
All Related Versions
This publication has 43 references indexed in Scilit:
- CIDEr: Consensus-based image description evaluation2015
- The Rijksmuseum Challenge2014
- Meteor Universal: Language Specific Translation Evaluation for Any Target Language2014
- The Stanford CoreNLP Natural Language Processing Toolkit2014
- Convolutional Neural Networks for Sentence Classification2014
- Recognizing Image Style2014
- An Optimal Assessment of Natural Language Student Input Using Word-to-Word Similarity Metrics2012
- Artistic Image Classification: An Analysis on the PRINTART Database2012
- Impressionism, expressionism, surrealismACM Transactions on Applied Perception, 2010
- Long Short-Term MemoryNeural Computation, 1997