Evaluating The Effects of Calibrated Popularity Bias Mitigation: A Field Study
Open Access
- 14 September 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1084-1089
Abstract
Despite their proven various benefits, Recommender Systems can cause or amplify certain undesired effects. In this paper, we focus on Popularity Bias, i.e., the tendency of a recommender system to utilize the effect of recommending popular items to the user. Prior research has studied the negative impact of this type of bias on individuals and society as a whole and proposed various approaches to mitigate this in various domains. However, almost all works adopted offline methodologies to evaluate the effectiveness of the proposed approaches. Unfortunately, such offline simulations can potentially be rather simplified and unable to capture the full picture. To contribute to this line of research and given a particular lack of knowledge about how debiasing approaches work not only offline, but online as well, we present in this paper the results of user study on a national broadcaster movie streaming platform in Norway, i.e., TV 2, following the A/B testing methodology. We deployed an effective mitigation approach for popularity bias, called Calibrated Popularity (CP), and monitored its performance in comparison to the platform’s existing collaborative filtering recommendation approach as a baseline over a period of almost four months. The results obtained from a large user base interacting in real-time with the recommendations indicate that the evaluated debiasing approach can be effective in addressing popularity bias while still maintaining the level of user interest and engagement.Keywords
Funding Information
- Research Council of Norway (309339)
This publication has 18 references indexed in Scilit:
- Using Stable Matching to Optimize the Balance between Accuracy and Diversity in Recommendation2020
- Controlling Popularity Bias in Learning-to-Rank Recommendation2017
- What recommenders recommend: an analysis of recommendation biases and possible countermeasuresUser Modeling and User-Adapted Interaction, 2015
- Escaping your comfort zone: A graph-based recommender system for finding novel recommendations among relevant itemsExpert Systems with Applications, 2015
- Recommending without short head2014
- Opinion-Based Collaborative Filtering to Solve Popularity Bias in Recommender Systems2013
- Challenging the long tail recommendationProceedings of the VLDB Endowment, 2012
- Item popularity and recommendation accuracy2011
- Understanding choice overload in recommender systems2010
- The long tail of recommender systems and how to leverage it2008