Multi-Task Neural Linear Bandit for Exploration in Recommender Systems
Open Access
- 24 August 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 5723-5730
Abstract
Exposure bias and its induced feedback loop effect are well-known problems in recommender systems. Exploration is believed to be the key to break such feedback loops. While classical contextual bandit algorithms such as Upper-Confidence-Bound and Thompson Sampling have been successful in addressing the exploration-exploitation trade-off in the single-task settings with one clear reward signal, modern recommender systems often leverage multiple rich sources of feedback such as clicks, likes, dislikes, shares, satisfaction survey responses, and employ multi-task learning in practice. It is unclear how one can incorporate exploration in the multi-task setup with different objectives. In this paper, we study an efficient bandit algorithm tailored to multi-task recommender systems, named Multi-task Neural Linear Bandit (mtNLB). In particular, we investigate efficient feature embeddings in the multi-task setups that could be used as contextual features in the Neural Linear Bandit, a contextual bandit algorithm that nicely combines the representation power from DNN and simplicity in uncertainty calculation from linear models. We further study cost-effective approximations of the uncertainty estimate and principled ways to incorporate uncertainty into the multi-task scoring of items. To showcase the efficacy of our proposed method, we conduct live experiments on a large-scale commercial recommendation platform that serves billions of users. We evaluate the quality of the uncertainty estimate and demonstrate its ability to improve exploration across the different dimensions of the reward signals in comparison to baseline approaches.Keywords
This publication has 26 references indexed in Scilit:
- A survey of recommender systems with multi-objective optimizationNeurocomputing, 2022
- Why I like it2018
- Explore, exploit, and explain2018
- Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts2018
- Short-Term Satisfaction and Long-Term Coverage2018
- Cold-Start Item and User Recommendation with Decoupled Completion and Transduction2015
- Item cold-start recommendations2014
- Pareto-efficient hybridization for multi-objective recommender systems2012
- A contextual-bandit approach to personalized news article recommendation2010
- Matrix Factorization Techniques for Recommender SystemsComputer, 2009