Social-sensed Image Aesthetics Assessment
Authors:
Open Access
- 31 October 2020
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 16 (3s) , 1-19
Abstract
Image aesthetics assessment aims to endow computers with the ability to judge the aesthetic values of images, and its potential has been recognized in a variety of applications. Most previous studies perform aesthetics assessment purely based on image content. However, given the fact that aesthetic perceiving is a human cognitive activity, it is necessary to consider users’ perception of an image when judging its aesthetic quality. In this article, we regard users’ social behavior as the reflection of their perception of images and harness these additional clues to improve image aesthetics assessment. Specifically, we first merge the raw social interactions between users and images into clusters as the social labels of images, so the collective social behavioral information associated with an image can be well represented over a structured and compact space. Then, we develop a novel deep multi-task network to jointly learn social labels in different modalities from social images and apply it to common web images. In this manner, our approach is readily generalized to web images without social behavioral information. Finally, we introduce a high-level fusion sub-network to the aesthetics model, in which the social and visual representations of images are well balanced for aesthetics assessment. Experimental results on two benchmark datasets well verify the effectiveness of our approach and highlight the benefits of different types of social behavioral information for image aesthetics assessment.Keywords
Funding Information
- National Natural Science Foundation of China (61701281 and 61876098)
- Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions
- National Key R8D Program of China (2018YFC0830100 and 2018YFC0830102)
- Natural Science Foundation of Shandong Province (ZR2017QF009)
This publication has 40 references indexed in Scilit:
- Rating Image Aesthetics Using Deep LearningIEEE Transactions on Multimedia, 2015
- What makes an image popular?2014
- Social-Sensed Image SearchACM Transactions on Information Systems, 2014
- Social event detection with interaction graph modeling2012
- The role of attractiveness in web image search2011
- Assessing the aesthetic quality of photographs using generic image descriptors2011
- Aesthetic quality classification of photographs based on color harmony2011
- Prediction of favourite photos using social, visual, and textual signals2010
- Photo and Video Quality Evaluation: Focusing on the Subject2008
- The cognitive impact of past behavior: Influences on beliefs, attitudes, and future behavioral decisions.Journal of Personality and Social Psychology, 2000