Semantic Sparse Recoding of Visual Content for Image Applications

Authors:

Abstract
This paper presents a new semantic sparse recoding method to generate more descriptive and robust representation of visual content for image applications. Although the visual bag-of-words (BOW) representation has been reported to achieve promising results in different image applications, its visual codebook is completely learnt from low-level visual features using quantization techniques and thus the so-called semantic gap remains unbridgeable. To handle such challenging issue, we utilize the annotations (predicted by algorithms or shared by users) of all the images to improve the original visual BOW representation. This is further formulated as a sparse coding problem so that the noise issue induced by the inaccurate quantization of visual features can also be handled to some extent. By developing an efficient sparse coding algorithm, we successfully generate a new visual BOW representation for image applications. Since such sparse coding has actually incorporated the high-level semantic information into the original visual codebook, we thus consider it as semantic sparse recoding of the visual content. Finally, we apply our semantic sparse recoding method to automatic image annotation and social image classification. The experimental results on several benchmark datasets show the promising performance of our semantic sparse recoding method in these two image applications.
Funding Information
  • National Natural Science Foundation of China (61202231, 61222307)
  • National Key Basic Research Program (973 program) of China (2014CB340403)
  • Beijing Natural Science Foundation of China (4132037)
  • Ph.D. Programs Foundation through the Ministry of Education, China (20120001120130)
  • Fundamental Research Funds for the Central Universities and the Research Funds through Renmin University of China, Beijing, China (14XNLF04)
  • IBM Faculty Award

This publication has 29 references indexed in Scilit: