Joint Optimization of Ranking and Calibration with Contextualized Hybrid Model
Open Access
- 4 August 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 4813-4822
Abstract
Despite the development of ranking optimization techniques, pointwise loss remains the dominating approach for click-through rate prediction. It can be attributed to the calibration ability of the pointwise loss since the prediction can be viewed as the click probability. In practice, a CTR prediction model is also commonly assessed with the ranking ability. To optimize the ranking ability, ranking loss (e.g., pairwise or listwise loss) can be adopted as they usually achieve better rankings than pointwise loss. Previous studies have experimented with a direct combination of the two losses to obtain the benefit from both losses and observed an improved performance. However, previous studies break the meaning of output logit as the click-through rate, which may lead to sub-optimal solutions. To address this issue, we propose an approach that can Jointly optimize the Ranking and Calibration abilities (JRC for short). JRC improves the ranking ability by contrasting the logit value for the sample with different labels and constrains the predicted probability to be a function of the logit subtraction. We further show that JRC consolidates the interpretation of logits, where the logits model the joint distribution. With such an interpretation, we prove that JRC approximately optimizes the contextualized hybrid discriminative-generative objective. Experiments on public and industrial datasets and online A/B testing show that our approach improves both ranking and calibration abilities. Since May 2022, JRC has been deployed on the display advertising platform of Alibaba and has obtained significant performance improvements.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Deep Interest Network for Click-Through Rate Prediction2018
- Optimized Cost per Click in Taobao Display Advertising2017
- Product-Based Neural Networks for User Response Prediction2016
- Wide & Deep Learning for Recommender Systems2016
- Practical Lessons from Predicting Clicks on Ads at Facebook2014
- Incremental Support Vector Learning for Ordinal RegressionIEEE Transactions on Neural Networks and Learning Systems, 2014
- Combined regression and ranking2010
- Semi-Supervised Learning2006
- An introduction to ROC analysisPattern Recognition Letters, 2006
- Cumulated gain-based evaluation of IR techniquesACM Transactions on Information Systems, 2002