Vocabulary-Informed Zero-Shot and Open-Set Learning
- 11 June 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 42 (12) , 3136-3152
Abstract
Despite significant progress in object categorization, in recent years, a number of important challenges remain; mainly, ability to learn from limited labeled data and ability to recognize object classes within large, potentially open, set of labels. Zero-shot learning is one way of addressing these challenges, but it has only been shown to work with limited sized class vocabularies and typically requires separation between supervised and unsupervised classes, allowing former to inform the latter but not vice versa. We propose the notion of vocabulary-informed learning to alleviate the above mentioned challenges and address problems of supervised, zero-shot, generalized zero-shot and open set recognition using a unified framework. Specifically, we propose a weighted maximum margin framework for semantic manifold-based recognition that incorporates distance constraints from (both supervised and unsupervised) vocabulary atoms. Distance constraints ensure that labeled samples are projected closer to their correct prototypes, in the embedding space, than to others. We illustrate that resulting model shows improvements in supervised, zero-shot, generalized zero-shot, and large open set recognition, with up to 310K class vocabulary on AwA and ImageNet datasets.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (61702108, 61622204)
- Science and Technology Commission of Shanghai Municipality (16JC1420400)
- Eastern Scholar (TP2017006)
- Shanghai Municipal Science and Technology Major Project (2017SHZDZX01, 2018SHZDZX01)
- ZJLab
This publication has 69 references indexed in Scilit:
- Probability Models for Open Set RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Attribute-Based Classification for Zero-Shot Visual Object CategorizationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Learning Multimodal Latent AttributesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human BrainNeuron, 2012
- Toward Open Set RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Attribute Learning for Understanding Unstructured Social Activity2012
- Hybrid Deterministic-Stochastic Methods for Data FittingSIAM Journal on Scientific Computing, 2012
- Using the forest to see the treesCommunications of the ACM, 2010
- Boosting the margin: a new explanation for the effectiveness of voting methodsThe Annals of Statistics, 1998
- Recognition-by-components: A theory of human image understanding.Psychological Review, 1987