Cross-Modal Learning to Rank via Latent Joint Representation
Authors:
- 12 February 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 24 (5) , 1497-1509
Abstract
Cross-modal ranking is a research topic that is imperative to many applications involving multimodal data. Discovering a joint representation for multimodal data and learning a ranking function are essential in order to boost the cross-media retrieval (i.e., image-query-text or text-query-image). In this paper, we propose an approach to discover the latent joint representation of pairs of multimodal data (e.g., pairs of an image query and a text document) via a conditional random field and structural learning in a listwise ranking manner. We call this approach cross-modal learning to rank via latent joint representation (CML 2 R). In CML 2 R, the correlations between multimodal data are captured in terms of their sharing hidden variables (e.g., topics), and a hidden-topic-driven discriminative ranking function is learned in a listwise ranking manner. The experiments show that the proposed approach achieves a good performance in cross-media retrieval and meanwhile has the capability to learn the discriminative representation of multimodal data.Keywords
Funding Information
- National Basic Research Program of China (2012CB316400)
- National Natural Science Foundation of China (61472353, 61402401)
- 863 Program (2012AA012505)
- China Knowledge Centre for Engineering Sciences and Technology
- Fundamental Research Funds for the Central Universities
- Zhejiang Provincial Natural Science Foundation of China (LQ14F010004, LY14F020027)
- U.S. National Science Foundation (CCF-1017828)
- Zhejiang Provincial Engineering Center on Media Data Cloud Processing and Analysis
This publication has 23 references indexed in Scilit:
- Probabilistic Word Selection via Topic ModelingIEEE Transactions on Knowledge and Data Engineering, 2014
- Multi-modal Mutual Topic Reinforce Modeling for Cross-media Retrieval2014
- The classification of multi-modal data with hidden conditional random fieldPattern Recognition Letters, 2014
- Click Prediction for Web Image Reranking Using Multimodal Sparse CodingIEEE Transactions on Image Processing, 2014
- A low rank structural large margin method for cross-modal ranking2013
- Learning to rank with (a lot of) word featuresInformation Retrieval Journal, 2009
- NUS-WIDE2009
- Learning to rank using gradient descent2005
- Canonical Correlation Analysis: An Overview with Application to Learning MethodsNeural Computation, 2004
- Optimizing search engines using clickthrough data2002