Benchmarking Representation Learning for Natural World Image Collections
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 12879-12888
Abstract
Recent progress in self-supervised learning has resulted in models that are capable of extracting rich representations from image collections without requiring any explicit label supervision. However, to date the vast majority of these approaches have restricted themselves to training on standard benchmark datasets such as ImageNet. We argue that fine-grained visual categorization problems, such as plant and animal species classification, provide an informative testbed for self-supervised learning. In order to facilitate progress in this area we present two new natural world visual classification datasets, iNat2021 and NeWT. The former consists of 2.7M images from 10k different species up-loaded by users of the citizen science application iNaturalist. We designed the latter, NeWT, in collaboration with domain experts with the aim of benchmarking the performance of representation learning algorithms on a suite of challenging natural world binary classification tasks that go beyond standard species classification. These two new datasets allow us to explore questions related to large-scale representation and transfer learning in the context of fine-grained categories. We provide a comprehensive analysis of feature extractors trained with and without supervision on ImageNet and iNat2021, shedding light on the strengths and weaknesses of different learned features across a diverse set of tasks. We find that features produced by standard supervised methods still outperform those produced by self-supervised approaches such as SimCLR. However, improved self-supervised learning methods are constantly being released and the iNat2021 and NeWT datasets are a valuable resource for tracking their progress.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- A large-scale car dataset for fine-grained categorization and verification2015
- Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Jointly Optimizing 3D Model Fitting and Fine-Grained Classification2014
- 3D Object Representations for Fine-Grained Categorization2013
- Biodiversity loss and its impact on humanityNature, 2012
- Dog Breed Classification Using Part Localization2012
- Leafsnap: A Computer Vision System for Automatic Plant Species Identification2012
- eBird: A citizen-based bird observation network in the biological sciencesBiological Conservation, 2009
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009