Optimization of Rank Losses for Image Retrieval
- 20 February 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (6) , 4317-4329
Abstract
In image retrieval, standard evaluation metrics rely on score ranking, e.g. average precision (AP), recall at k (R@k), normalized discounted cumulative gain (NDCG). In this work, we introduce a general framework for robust and decomposable rank losses optimization. It addresses two major challenges for end-to-end training of deep neural networks with rank losses: non-differentiability and non-decomposability. Firstly, we propose a general surrogate for ranking operator, SupRank, that is amenable to stochastic gradient descent. It provides an upperbound for rank losses and ensures robust training. Secondly, we use a simple yet effective loss function to reduce the decomposability gap between the averaged batch approximation of ranking losses and their values on the whole training set. We apply our framework to two standard metrics for image retrieval: AP and R@k. Additionally, we apply our framework to hierarchical image retrieval. We introduce an extension of AP, the hierarchical average precision ${\mathcal {H}}$, and optimize it as well as the NDCG. Finally, we create the first hierarchical landmarks retrieval dataset. We use a semi-automatic pipeline to create hierarchical labels, extending the large scale Google Landmarks v2 dataset. The hierarchical dataset is publicly available at github.com/cvdfoundation/google-landmark. Code is available at github.com/elias-ramzi/SupRank.Keywords
All Related Versions
Funding Information
- AHEAD ANR (ANR-20-THIA-0002)
This publication has 61 references indexed in Scilit:
- Dynamic Metric Learning with Cross-Level Concept Distillation2022
- Hierarchical Average Precision Training for Pertinent Image Retrieval2022
- ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis2020
- Hard Negative Examples are Hard, but Useful2020
- A Unifying Mutual Information View of Metric Learning: Cross-Entropy vs. Pairwise Losses2020
- Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval2020
- Deep Metric Learning with Hierarchical Triplet Loss2018
- CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples2016
- The foundation of the concept of relevanceJournal of the American Society for Information Science and Technology, 2009
- Using graded relevance assessments in IR evaluationJournal of the American Society for Information Science and Technology, 2002