How scales influence user rating behaviour in recommender systems
- 9 May 2017
- journal article
- research article
- Published by Taylor & Francis in Behaviour & Information Technology
- Vol. 36 (10) , 985-1004
Abstract
Many websites allow users to rate items and share their ratings with others, for social or personalisation purposes. In recommender systems in particular, personalised suggestions are generated by predicting ratings for items that users are unaware of, based on the ratings users provided for other items. Explicit user ratings are collected by means of graphical widgets referred to as ‘rating scales’. Each system or website normally uses a specific rating scale, in many cases differing from scales used by other systems in their granularity, visual metaphor, numbering or availability of a neutral position. While many works in the field of survey design reported on the effects of rating scales on user ratings, these, however, are normally regarded as neutral tools when it comes to recommender systems. In this paper, we challenge this view and provide new empirical information about the impact of rating scales on user ratings, presenting the results of three new studies carried out in different domains. Based on these results, we demonstrate that a static mathematical mapping is not the best method to compare ratings coming from scales with different features, and suggest when it is possible to use linear functions instead.Keywords
This publication has 20 references indexed in Scilit:
- The Effects of the Direction of Rating Scales on Survey Responses in a Telephone SurveyPublic Opinion Quarterly, 2015
- A Study on User Preferential Choices about Rating ScalesInternational Journal of Technology and Human Interaction, 2015
- Navigational User Interface Elements on the Left Side: Intuition of Designers or Experimental Evidence?2011
- The effect of rating scale format on response styles: The number of response categories and response category labelsInternational Journal of Research in Marketing, 2010
- Relative Ranking – A Biased Rating2009
- The Use of Different Happiness Rating Scales: Bias and Comparison Problem?Social Indicators Research, 2007
- Color, Labels, and Interpretive Heuristics for Response ScalesPublic Opinion Quarterly, 2007
- Impact of the Number of Response Categories and Anchor Labels on Coefficient Alpha and Test-Retest ReliabilityEducational and Psychological Measurement, 2004
- Designing Usable Interfaces for TV Recommender Systems2004
- Evaluating collaborative filtering recommender systemsACM Transactions on Information Systems, 2004