Understanding Art through Multi-Modal Retrieval in Paintings
Preprint
Abstract
In computer vision, visual arts are often studied from a purely aesthetics perspective, mostly by analysing the visual appearance of an artistic reproduction to infer its style, its author, or its representative features. In this work, however, we explore art from both a visual and a language perspective. Our aim is to bridge the gap between the visual appearance of an artwork and its underlying meaning, by jointly analysing its aesthetics and its semantics. We introduce the use of multi-modal techniques in the field of automatic art analysis by 1) collecting a multi-modal dataset with fine-art paintings and comments, and 2) exploring robust visual and textual representations in artistic images.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Context-Aware Embeddings for Automatic Art Analysis2019
- A Style-Aware Content Loss for Real-Time HD Style Transfer2018
- Sketching with Style: Visual Search with Sketches and Aesthetic Context2017
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image AnnotationsInternational Journal of Computer Vision, 2017
- Ceci n'est pas une pipe: A deep convolutional network for fine-art paintings classification2016
- Deep Residual Learning for Image Recognition2016
- Classification of Artistic Styles Using Binarized Features Derived from a Deep Neural Network2015
- Painting-91: a large scale database for computational painting categorizationMachine Vision and Applications, 2014
- Microsoft COCO: Common Objects in Context2014
- Artistic Image Classification: An Analysis on the PRINTART Database2012