Hierarchical Average Precision Training for Pertinent Image Retrieval
Preprint
Abstract
Image Retrieval is commonly evaluated with Average Precision (AP) or Recall@k. Yet, those metrics, are limited to binary labels and do not take into account errors' severity. This paper introduces a new hierarchical AP training method for pertinent image retrieval (HAP-PIER). HAPPIER is based on a new H-AP metric, which leverages a concept hierarchy to refine AP by integrating errors' importance and better evaluate rankings. To train deep models with H-AP, we carefully study the problem's structure and design a smooth lower bound surrogate combined with a clustering loss that ensures consistent ordering. Extensive experiments on 6 datasets show that HAPPIER significantly outperforms state-of-the-art methods for hierarchical retrieval, while being on par with the latest approaches when evaluating fine-grained ranking performances. Finally, we show that HAPPIER leads to better organization of the embedding space, and prevents most severe failure cases of non-hierarchical methods. Our code is publicly available at: https://github.com/elias-ramzi/HAPPIER.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Array programming with NumPyNature, 2020
- GPU accelerated t-distributed stochastic neighbor embeddingJournal of Parallel and Distributed Computing, 2019
- Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking2018
- CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples2016
- Learning a Distance Metric from Relative Comparisons between Quadruplets of ImagesInternational Journal of Computer Vision, 2016
- Deep Residual Learning for Image Recognition2016
- Model Based Comparison of Discounted Cumulative Gain and Average PrecisionJournal of Discrete Algorithms, 2013
- A general approximation framework for direct optimization of information retrieval measuresInformation Retrieval Journal, 2009
- ImageNet: A large-scale hierarchical image database2009
- Using graded relevance assessments in IR evaluationJournal of the American Society for Information Science and Technology, 2002