Comparisons Instead of Ratings: Towards More Stable Preferences
Authors:
- Nicolas JonesNJNicolas Jones
- Armelle BrunABArmelle Brun
- Anne BoyerABAnne Boyer
- 1 August 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1, 451-456
Abstract
More and more personalization systems are emerging to reduce the information overload of the Web. As a result, it has become vital to model users' preferences accurately. Our focus lies in the quality of users' expressed preferences, in terms of reliability and stability through time. Today, users are often brought to express their preferences through ratings on a multi-point scale. However, several studies have highlighted problems with ratings. We propose a new preference modality whereby users compare items two-by two ("I prefer x to y").This initial work on comparisons shows that users are in favor of this new preference mechanism and that comparisons are almost 20% more stable over time than those conveyed through ratings, thus more reliable. These encouraging findings let us think that comparisons may lead to a better user modeling and an increase in the quality of personalization services, such as recommender systems.Keywords
This publication has 8 references indexed in Scilit:
- An Exploratory Work in Using Comparisons Instead of Ratings2011
- The wisdom of the few2009
- Evaluation measures for preference judgments2008
- Is seeing believing?2003
- Mining the peanut gallery2003
- Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferencesActa Psychologica, 2000
- Relative effectiveness and validity of mood induction procedures: a meta-analysisEuropean Journal of Social Psychology, 1996
- GroupLens1994