A Shared Multi-Attention Framework for Multi-Label Zero-Shot Learning
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8773-8783
Abstract
In this work, we develop a shared multi-attention model for multi-label zero-shot learning. We argue that designing attention mechanism for recognizing multiple seen and unseen labels in an image is a non-trivial task as there is no training signal to localize unseen labels and an image only contains a few present labels that need attentions out of thousands of possible labels. Therefore, instead of generating attentions for unseen labels which have unknown behaviors and could focus on irrelevant regions due to the lack of any training sample, we let the unseen labels select among a set of shared attentions which are trained to be label-agnostic and to focus on only relevant/foreground regions through our novel loss. Finally, we learn a compatibility function to distinguish labels based on the selected attention. We further propose a novel loss function that consists of three components guiding the attention to focus on diverse and relevant image regions while utilizing all attention features. By extensive experiments, we show that our method improves the state of the art by 2.9% and 1.4% F1 score on the NUS-WIDE and the large scale Open Images datasets, respectively.Keywords
This publication has 32 references indexed in Scilit:
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels2016
- Fast Zero-Shot Image Tagging2016
- Label-Embedding for Image ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Is object localization for free? - Weakly-supervised learning with convolutional neural networks2015
- Deep visual-semantic alignments for generating image descriptions2015
- COSTA: Co-Occurrence Statistics for Zero-Shot Classification2014
- Glove: Global Vectors for Word Representation2014
- NUS-WIDE2009
- Multi-Label ClassificationInternational Journal of Data Warehousing and Mining, 2007