"Of All Things the Measure Is Man" : Automatic Classification of Emotions and Inter-Labeler Consistency
- 11 October 2006
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1 (15206149) , 317-320
Abstract
In traditional classification problems, the reference needed for training a classifier is given and considered to be absolutely correct. However, this does not apply to all tasks. In emotion recognition in non-acted speech, for instance, one often does not know which emotion was really intended by the speaker. Hence, the data is annotated by a group of human labelers who do not agree on one common class in most cases. Often, similar classes are confused systematically. We propose a new entropy-based method to evaluate classification results taking into account these systematic confusions. We can show that a classifier which achieves a recognition rate of "only" about 60 % on a four-class-problem performs as well as our five human labelers on average.Keywords
This publication has 3 references indexed in Scilit:
- How to find trouble in communicationSpeech Communication, 2003
- Normalization of non-standard wordsComputer Speech & Language, 2001
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948