Social-sensed Image Aesthetics Assessment

Authors:

Abstract
Image aesthetics assessment aims to endow computers with the ability to judge the aesthetic values of images, and its potential has been recognized in a variety of applications. Most previous studies perform aesthetics assessment purely based on image content. However, given the fact that aesthetic perceiving is a human cognitive activity, it is necessary to consider users’ perception of an image when judging its aesthetic quality. In this article, we regard users’ social behavior as the reflection of their perception of images and harness these additional clues to improve image aesthetics assessment. Specifically, we first merge the raw social interactions between users and images into clusters as the social labels of images, so the collective social behavioral information associated with an image can be well represented over a structured and compact space. Then, we develop a novel deep multi-task network to jointly learn social labels in different modalities from social images and apply it to common web images. In this manner, our approach is readily generalized to web images without social behavioral information. Finally, we introduce a high-level fusion sub-network to the aesthetics model, in which the social and visual representations of images are well balanced for aesthetics assessment. Experimental results on two benchmark datasets well verify the effectiveness of our approach and highlight the benefits of different types of social behavioral information for image aesthetics assessment.
Funding Information
  • National Natural Science Foundation of China (61701281 and 61876098)
  • Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions
  • National Key R8D Program of China (2018YFC0830100 and 2018YFC0830102)
  • Natural Science Foundation of Shandong Province (ZR2017QF009)

This publication has 40 references indexed in Scilit: