Loss Functions for Top-k Error: Analysis and Insights
Authors:
- Maksim LapinMLMaksim Lapin
- Matthias HeinMHMatthias Hein
- Bernt SchieleBSBernt Schiele
- 12 December 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1468-1477
Abstract
In order to push the performance on realistic computer vision tasks, the number of classes in modern benchmark datasets has significantly increased in recent years. This increase in the number of classes comes along with increased ambiguity between the class labels, raising the question if top-1 error is the right performance measure. In this paper, we provide an extensive comparison and evaluation of established multiclass methods comparing their top-k performance both from a practical as well as from a theoretical perspective. Moreover, we introduce novel top-k loss functions as modifications of the softmax and the multiclass SVM losses and provide efficient optimization schemes for them. In the experiments, we compare on various datasets all of the proposed and established methods for top-k error optimization. An interesting insight of this paper is that the softmax loss yields competitive top-k performance for all k simultaneously. For a specific top-k error, our new top-k losses lead typically to further improvements while being faster to train than the softmax.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Precise and fast computation of Lambert-functions without transcendental function evaluationsJournal of Computational and Applied Mathematics, 2013
- Smoothing and First Order Methods: A Unified FrameworkSIAM Journal on Optimization, 2012
- The Infinite Push: A New Support Vector Ranking Algorithm that Directly Optimizes Accuracy at the Absolute Top of the List2011
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Material perception: What can you see in a brief glance?Journal of Vision, 2009
- Learning Deep Architectures for AIFoundations and Trends in Machine Learning, 2009
- Convexity, Classification, and Risk BoundsJournal of the American Statistical Association, 2006
- Smooth minimization of non-smooth functionsMathematical Programming, 2004
- A comparison of methods for multiclass support vector machinesIEEE Transactions on Neural Networks, 2002
- On the LambertW functionAdvances in Computational Mathematics, 1996