Rethinking User Retention Modeling in Recommendation
Open Access
- 2 March 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 44 (3) , 1-35
Abstract
Recommendations usually focus on immediate accuracy metrics like Click-Through Rate (CTR), ignoring user long-term metrics. User retention, which reflects the percentage of today’s users who will return to the system in the next few days, should be paid more attention to. However, most existing methods did not focus on user retention, since their complexity and uncertainty make it extremely hard to discover why a user will or will not return to a system. Recently, a few pioneers have optimized user retention, focusing solely on accuracy without delving into its underlying rationale. This is primarily due to the absence of explicit supervised signals. In this work, we design a Behavior-wise Contrastive Multi-Instance Learning (BCMIL) module, which jointly models clicked and impressed items to capture interpretable user retention. Specifically, we conduct in-depth analyses in real-world scenarios to discover implicit retention-related supervised signals. To model these signals, we design a Forward Supervised Signals Extractor (FSSE) that utilizes a heterogeneous graph, enhancing the reliability of user retention. To mitigate randomness and uncertainty, we propose a Backward Supervised Signals Stabilizer (BSSS) that utilizes overlooked label-part behaviors within each training window to retrospectively guide the training process. Offline and online evaluations of an industrial system verify the effectiveness of our methods.Keywords
This publication has 43 references indexed in Scilit:
- BiGAN: Collaborative Filtering with Bidirectional Generative Adversarial Networks2020
- Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems2019
- DRN2018
- Returning is Believing2017
- IRGAN2017
- User Personalized Satisfaction Prediction via Multiple Instance Deep Learning2017
- Using Implicit Preference Relations to Improve Recommender SystemsJournal on Data Semantics, 2016
- Beyond clicks2014
- Timespent based models for predicting user retention2013
- Churn prediction in new users of Yahoo! answers2012