Improving Facial Attribute Prediction Using Semantic Segmentation
Authors:
- Mahdi M. KalayehMKMahdi M. Kalayeh
- Boqing GongBGBoqing Gong
- Mubarak ShahMSMubarak Shah
- 1 July 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 4227-4235
Abstract
Attributes are semantically meaningful characteristics whose applicability widely crosses category boundaries. They are particularly important in describing and recognizing concepts where no explicit training example is given, e.g., zero-shot learning. Additionally, since attributes are human describable, they can be used for efficient human-computer interaction. In this paper, we propose to employ semantic segmentation to improve facial attribute prediction. The core idea lies in the fact that many facial attributes describe local properties. In other words, the probability of an attribute to appear in a face image is far from being uniform in the spatial domain. We build our facial attribute prediction model jointly with a deep semantic segmentation network. This harnesses the localization cues learned by the semantic segmentation to guide the attention of the attribute prediction to the regions where different attributes naturally show up. As a result of this approach, in addition to recognition, we are able to localize the attributes, despite merely having access to image level labels (weak supervision) during training. We evaluate our proposed method on CelebA and LFWA datasets and achieve superior results to the prior arts. Furthermore, we show that in the reverse problem, semantic face parsing improves when facial attributes are available. That reaffirms the need to jointly model these two interconnected tasks.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes2016
- Weakly Supervised Deep Detection Networks2016
- Learning Deep Representation for Imbalanced Classification2016
- Deep Learning Face Attributes in the Wild2015
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Learning Deconvolution Network for Semantic Segmentation2015
- Actions and Attributes from Wholes and Parts2015
- Describing people: A poselet-based approach to attribute classification2011
- Attribute and simile classifiers for face verification2009
- FaceTracer: A Search Engine for Large Collections of Images with Faces2008