Nonlinear Bandits Exploration for Recommendations
Open Access
- 14 September 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1054-1057
Abstract
The paradigm of framing recommendations as (sequential) decision-making processes has gained significant interest. To achieve long-term user satisfaction, these interactive systems need to strike a balance between exploitation (recommending high-reward items) and exploration (exploring uncertain regions for potentially better items). Classical bandit algorithms like Upper-Confidence-Bound and Thompson Sampling, and their contextual extensions with linear payoffs have exhibited strong theoretical guarantees and empirical success in managing the exploration-exploitation trade-off. Building efficient exploration-based systems for deep neural network powered real-world, large-scale industrial recommender systems remains under studied. In addition, these systems are often multi-stage, multi-objective and response time sensitive. In this talk, we share our experience in addressing these challenges in building exploration based industrial recommender systems. Specifically, we adopt the Neural Linear Bandit algorithm, which effectively combines the representation power of deep neural networks, with the simplicity of linear bandits to incorporate exploration in DNN based recommender systems. We introduce exploration capability to both the nomination and ranking stage of the industrial recommender system. In the context of the ranking stage, we delve into the extension of this algorithm to accommodate the multi-task setup, enabling exploration in systems with multiple objectives. Moving on to the nomination stage, we will address the development of efficient bandit algorithms tailored to factorized bi-linear models. These algorithms play a crucial role in facilitating maximum inner product search, which is commonly employed in large-scale retrieval systems. We validate our algorithms and present findings from real-world live experiments.Keywords
This publication has 13 references indexed in Scilit:
- Mixed Negative Sampling for Learning Two-tower Neural Networks in Recommendations2020
- Recommending what video to watch next2019
- Top-K Off-Policy Correction for a REINFORCE Recommender System2019
- Degenerate Feedback Loops in Recommender Systems2019
- Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts2018
- Performance of recommender algorithms on top-n recommendation tasks2010
- A contextual-bandit approach to personalized news article recommendation2010
- Matrix Factorization Techniques for Recommender SystemsComputer, 2009
- Item-based collaborative filtering recommendation algorithms2001
- ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLESBiometrika, 1933