Addressing Popularity Bias in Third-Party Library Recommendations Using LLMs
Preprint
Abstract
Recommender systems for software engineering (RSSE) play a crucial role in automating development tasks by providing relevant suggestions according to the developer's context. However, they suffer from the so-called popularity bias, i.e., the phenomenon of recommending popular items that might be irrelevant to the current task. In particular, the long-tail effect can hamper the system's performance in terms of accuracy, thus leading to false positives in the provided recommendations. Foundation models are the most advanced generative AI-based models that achieve relevant results in several SE tasks. This paper aims to investigate the capability of large language models (LLMs) to address the popularity bias in recommender systems of third-party libraries (TPLs). We conduct an ablation study experimenting with state-of-the-art techniques to mitigate the popularity bias, including fine-tuning and popularity penalty mechanisms. Our findings reveal that the considered LLMs cannot address the popularity bias in TPL recommenders, even though fine-tuning and post-processing penalty mechanism contributes to increasing the overall diversity of the provided recommendations. In addition, we discuss the limitations of LLMs in this context and suggest potential improvements to address the popularity bias in TPL recommenders, thus paving the way for additional experiments in this direction.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- A survey on popularity bias in recommender systemsUser Modeling and User-Adapted Interaction, 2024
- Fairness-aware machine learning engineering: how far are we?Empirical Software Engineering, 2023
- EqBal-RS: Mitigating popularity bias in recommender systemsJournal of Intelligent Information Systems, 2023
- Fairness in recommender systems: research landscape and future directionsUser Modeling and User-Adapted Interaction, 2023
- Debiaser for Multiple Variables to enhance fairness in classification tasksInformation Processing & Management, 2022
- Algorithmic fairness datasets: the story so farData Mining and Knowledge Discovery, 2022
- Development of recommendation systems for software engineering: the CROSSMINER experienceEmpirical Software Engineering, 2021
- Automatic Fairness Testing of Machine Learning Models2020
- What’s Spain’s Paris? Mining analogical libraries from Q&A discussionsEmpirical Software Engineering, 2018
- Recommendation Systems in Software Engineering2014