Semi-Supervised Fine-Grained Image Categorization Using Transfer Learning With Hierarchical Multi-Scale Adversarial Networks
Open Access
- 12 August 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 118650-118668
Abstract
Fine-grained image categorization is still a challenging computer vision problem in recent years. Most of existing methods highly rely on massive labeled data which are scarce in many real world applications. It should also be noticed that progressive learning demands of existing data is very common today. That is, we may pay attention to more fine-grained information (like arctic tern, black tern, buttercup or tulip) in an existing data set with labels like “bird” and “flower”. It is reasonable to believe that the existing labels and model with transferable knowledge would be helpful to another related but different, fine-grained recognition task. In this context, an improved transfer deep learning approach with hierarchical multi-adversarial networks is proposed in this paper. With this approach, cross domain features are extracted by advanced deep encoders coarsely. After that, we annotate a small amount of images in the target domain, thereby creating the “active labels” which can provide instructions for adversarial learning. Then, the GAN-based hierarchical model is utilized to select cross domain categories and enhance related features so as to facilitate an effective transfer. In order to exploit useful local features, a novel adaptive attention mechanism, Region Adversarial Network (RAN) which can select attention regions during adversarial learning and generate valuable fine-grained features, is introduced in the article. We call the proposed hierarchical framework “Attentional Multi-Adversarial Networks (AMAN)” . Experimental results show that AMAN is able to augment cross domain features well-directly and build an effective classifier for fine-grained categorization in the target domain with fewer training samples and higher accuracies.Keywords
Funding Information
- Key Research and Development Program of Ningxia Hui Autonomous Region (2019BFG02009)
- Ministry of Information Industry of the People's Republic of China
- National Natural Science Foundation of China (61801019)
This publication has 41 references indexed in Scilit:
- Attention to Scale: Scale-Aware Semantic Image Segmentation2016
- Embedding Label Structures for Fine-Grained Feature Representation2016
- Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?IEEE Transactions on Medical Imaging, 2016
- Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks2015
- CNN Features Off-the-Shelf: An Astounding Baseline for Recognition2014
- The Ignorant Led by the Blind: A Hybrid Human–Machine Vision System for Fine-Grained CategorizationInternational Journal of Computer Vision, 2014
- 3D Object Representations for Fine-Grained Categorization2013
- POOF: Part-Based One-vs.-One Features for Fine-Grained Categorization, Face Verification, and Attribute Estimation2013
- Selective Search for Object RecognitionInternational Journal of Computer Vision, 2013
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009