A loss function for classification based on a robust similarity metric
Authors:
- 1 July 2010
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21614393,p. 1-6
Abstract
We present a margin-based loss function for classification, inspired by the recently proposed similarity measure called correntropy. We show that correntropy induces a nonconvex loss function that is a closer approximation to the misclassification loss (ideal 0-1 loss). We show that the discriminant function obtained by optimizing the proposed loss function using a neural network is insensitive to outliers and has better generalization performance as compared to using the squared loss function which is common in neural network classifiers. The proposed method of training classifiers is a practical way of obtaining better results on real world classification problems, that uses a simple gradient based online training procedure for minimizing the empirical risk.Keywords
This publication has 10 references indexed in Scilit:
- Using Correntropy as a cost function in linear adaptive filters2009
- The correntropy MACE filterPattern Recognition, 2009
- A Pitch Detector Based on a Generalized Correlation FunctionIEEE Transactions on Audio, Speech, and Language Processing, 2008
- Correntropy: Properties and Applications in Non-Gaussian Signal ProcessingIEEE Transactions on Signal Processing, 2007
- Scaling Learning Algorithms toward AI2007
- Generalized correlation function: definition, properties, and application to blind equalizationIEEE Transactions on Signal Processing, 2006
- Convexity, Classification, and Risk BoundsJournal of the American Statistical Association, 2006
- A Tutorial on Support Vector Machines for Pattern RecognitionData Mining and Knowledge Discovery, 1998
- The Nature of Statistical Learning Theory1995
- On Estimation of a Probability Density Function and ModeThe Annals of Mathematical Statistics, 1962