Reinforcing User Retention in a Billion Scale Short Video Recommender System
Open Access
- 30 April 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 421-426
Abstract
Recently, short video platforms have achieved rapid user growth by recommending interesting content to users. The objective of the recommendation is to optimize user retention, thereby driving the growth of DAU (Daily Active Users). Retention is a long-term feedback after multiple interactions of users and the system, and it is hard to decompose retention reward to each item or a list of items. Thus traditional point-wise and list-wise models are not able to optimize retention. In this paper, we choose reinforcement learning methods to optimize the retention as they are designed to maximize the long-term performance. We formulate the problem as an infinite-horizon request-based Markov Decision Process, and our objective is to minimize the accumulated time interval of multiple sessions, which is equal to improving the app open frequency and user retention. However, current reinforcement learning algorithms can not be directly applied in this setting due to uncertainty, bias, and long delay time incurred by the properties of user retention. We propose a novel method, dubbed RLUR, to address the aforementioned challenges. Both offline and live experiments show that RLUR can significantly improve user retention. RLUR has been fully launched in Kuaishou app for a long time, and achieves consistent performance improvement on user retention and DAU.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems2019
- Reinforcement Knowledge Graph Reasoning for Explainable Recommendation2019
- A Deep Reinforcement Learning Approach to Proactive Content Pushing and Recommendation for Mobile UsersIEEE Access, 2019
- Stabilizing Reinforcement Learning in Dynamic Environment with Application to Online Recommendation2018
- Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning2018
- Returning is Believing2017
- Curiosity-Driven Exploration by Self-Supervised Prediction2017
- Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach2016
- Deep Neural Networks for YouTube Recommendations2016
- A Markov inequality in several dimensionsJournal of Approximation Theory, 1974