Discovering Visual Patterns in Art Collections With Spatially-Consistent Feature Learning
Authors:
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9270-9279
Abstract
Our goal in this paper is to discover near duplicate patterns in large collections of artworks. This is harder than standard instance mining due to differences in the artistic media (oil, pastel, drawing, etc), and imperfections inherent in the copying process. Our key technical insight is to adapt a standard deep feature to this task by fine-tuning it on the specific art collection using self-supervised learning. More specifically, spatial consistency between neighbouring feature matches is used as supervisory fine-tuning signal. The adapted feature leads to more accurate style invariant matching, and can be used with a standard discovery approach, based on geometric verification, to identify duplicate patterns in the dataset. The approach is evaluated on several different datasets and shows surprisingly good qualitative discovery results. For quantitative evaluation of the method, we annotated 273 near duplicate details in a dataset of 1587 artworks attributed to Jan Brueghel and his workshop. Beyond artworks, we also demonstrate improvement on localization on the Oxford5K photo dataset as well as on historical photograph localization on the Large Time Lags Location (LTLL) dataset.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Location recognition over large time lagsComputer Vision and Image Understanding, 2015
- Challenges in Content-Based Image Indexing of Cultural Heritage Collections: Support vector machine active learning with applications to text classificationIEEE Signal Processing Magazine, 2015
- The Rijksmuseum Challenge2014
- Painting-to-3D model alignment via discriminative visual elementsACM Transactions on Graphics, 2014
- Of Gods and Goats: Weakly Supervised Learning of Figurative Art2013
- What makes Paris look like Paris?ACM Transactions on Graphics, 2012
- Unsupervised Discovery of Mid-Level Discriminative Patches2012
- Data-driven visual similarity for cross-domain image matching2011
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Image analogies2001