Towards decrypting attractiveness via multi-modality cues
Authors:
Open Access
- 19 August 2013
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 9 (4) , 1-20
Abstract
Decrypting the secret of beauty or attractiveness has been the pursuit of artists and philosophers for centuries. To date, the computational model for attractiveness estimation has been actively explored in computer vision and multimedia community, yet with the focus mainly on facial features. In this article, we conduct a comprehensive study on female attractiveness conveyed by single/multiple modalities of cues, that is, face, dressing and/or voice, and aim to discover how different modalities individually and collectively affect the human sense of beauty. To extensively investigate the problem, we collect the Multi-Modality Beauty (M2B) dataset, which is annotated with attractiveness levels converted from manualk-wise ratings and semantic attributes of different modalities. Inspired by the common consensus that middle-level attribute prediction can assist higher-level computer vision tasks, we manually labeled many attributes for each modality. Next, a tri-layer Dual-supervised Feature-Attribute-Task (DFAT) network is proposed to jointly learn the attribute model and attractiveness model of single/multiple modalities. To remedy possible loss of information caused by incomplete manual attributes, we also propose a novel Latent Dual-supervised Feature-Attribute-Task (LDFAT) network, where latent attributes are combined with manual attributes to contribute to the final attractiveness estimation. The extensive experimental evaluations on the collected M2B dataset well demonstrate the effectiveness of the proposed DFAT and LDFAT networks for female attractiveness prediction.Keywords
Funding Information
- National Research Foundation-Prime Minister's office, Republic of Singapore
- Human Sixth Sense Programme at the Advanced Digital Sciences Center from Singapore's Agency for Science, Technology and Research
This publication has 27 references indexed in Scilit:
- New “golden” ratios for facial beautyVision Research, 2010
- Sex-Dimorphic Face Shape Preference in Heterosexual and Homosexual Men and WomenArchives of Sexual Behavior, 2009
- An Ingroup Advantage for Confidence in Emotion Recognition Judgments: The Moderating Effect of Familiarity With the Expressions of Outgroup MembersPersonality and Social Psychology Bulletin, 2006
- Facial Attractiveness: Beauty and the MachineNeural Computation, 2006
- Shape matching and object recognition using shape contextsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- All That Glitters: A Review of Psychological Research on the Aesthetics of the Golden SectionPerception, 1995
- Active Shape Models-Their Training and ApplicationComputer Vision and Image Understanding, 1995
- Article Commentary: Averaged Faces Are Attractive, but Very Attractive Faces Are Not AveragePsychological Science, 1991
- Effects of Clothing Attractiveness on PerceptionsHome Economics Research Journal, 1990
- Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filtersJournal of the Optical Society of America A, 1985