Automatically annotating the MIR Flickr dataset
Authors:
Open Access
- 29 March 2010
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 547-556
Abstract
The availability of a large, freely redistributable set of high-quality annotated images is critical to allowing researchers in the area of automatic annotation, generic object recognition and concept detection to compare results. The recent introduction of the MIR Flickr dataset allows researchers such access. A dataset by itself is not enough, and a set of repeatable guidelines for performing evaluations that are comparable is required. In many cases it also is useful to compare the machine-learning components of different automatic annotation techniques using a common set of image features. This paper seeks to provide a solid, repeatable methodology and protocol for performing evaluations of automatic annotation software using the MIR Flickr dataset together with freely available tools for measuring performance in a controlled manner. This protocol is demonstrated through a set of experiments using a "semantic space" auto-annotator previously developed by the authors, in combination with a set of visual term features for the images that has been made publicly available for download. The paper also discusses how much training data is required to train the semantic space annotator with the MIR Flickr dataset. It is the hope of the authors that researchers will adopt this methodology and produce results from their own annotators that can be directly compared to those presented in this work.Keywords
This publication has 17 references indexed in Scilit:
- Performance evaluation of local colour invariantsComputer Vision and Image Understanding, 2009
- Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive StudyInternational Journal of Computer Vision, 2006
- A Linear-Algebraic Technique with an Application in Semantic Image RetrievalLecture Notes in Computer Science, 2006
- On Image Retrieval Using Salient Regions with Vector-Spaces and Latent Semantics2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Evaluation of salient point techniquesImage and Vision Computing, 2003
- A performance evaluation of local descriptors2003
- The Truth about Corel - Evaluation in Image Retrieval2002
- Similarity of color images1995
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990