Ad Recommendation Systems for Life-Time Value Optimization
Authors:
Open Access
- 18 May 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1305-1310
Abstract
The main objective in the ad recommendation problem is to find a strategy that, for each visitor of the website, selects the ad that has the highest probability of being clicked. This strategy could be computed using supervised learning or contextual bandit algorithms, which treat two visits of the same user as two separate independent visitors, and thus, optimize greedily for a single step into the future. Another approach would be to use reinforcement learning (RL) methods, which differentiate between two visits of the same user and two different visitors, and thus, optimizes for multiple steps into the future or the life-time value (LTV) of a customer. While greedy methods have been well-studied, the LTV approach is still in its infancy, mainly due to two fundamental challenges: how to compute a good LTV strategy and how to evaluate a solution using historical data to ensure its \"safety\" before deployment. In this paper, we tackle both of these challenges by proposing to use a family of off-policy evaluation techniques with statistical guarantees about the performance of a new strategy. We apply these methods to a real ad recommendation problem, both for evaluating the final performance and for optimizing the parameters of the RL algorithm. Our results show that our LTV optimization algorithm equipped with these off-policy evaluation techniques outperforms the greedy approaches. They also give fundamental insights on the difference between the click through rate (CTR) and LTV metrics for performance evaluation in the ad recommendation problem.Keywords
This publication has 12 references indexed in Scilit:
- A contextual-bandit approach to personalized news article recommendation2010
- The 2005 ISMS Practice Prize Winner—Customer Equity and Lifetime Management (CELM) Finnair Case StudyMarketing Science, 2007
- Joint optimization of customer segmentation and marketing policy to maximize long-term profitabilityExpert Systems with Applications, 2004
- Prediction of Coronary Heart Disease in Middle-Aged Adults With DiabetesDiabetes Care, 2003
- Coronary heart disease risk prediction in the Atherosclerosis Risk in Communities (ARIC) studyJournal of Clinical Epidemiology, 2003
- Modern Applied Statistics with S2002
- Random ForestsMachine Learning, 2001
- Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticiansStatistics in Medicine, 2000
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Better Bootstrap Confidence IntervalsJournal of the American Statistical Association, 1987