Shashank Gupta
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Shashank Gupta has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Harrie Oosterhuis
University of Amsterdam, Amsterdam, Netherlands
7 shared publications - Maarten de Rijke
University of Amsterdam, Amsterdam, Netherlands
5 shared publications - Philipp Hager
University of Amsterdam, Amsterdam, Netherlands
3 shared publications - Ali Vardasbi
Spotify, Amsterdam, Netherlands
2 shared publications - Jin Huang
University of Cambridge, Cambridge, United Kingdom
2 shared publications - Olivier Jeunen
Independent Researcher, Antwerp, Belgium
2 shared publications
Latest Publications
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3828-3832
Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3828-3832
Open AccessAdditive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation
- Open Access
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 177-182
Towards Two-Stage Counterfactual Learning to Rank
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 177-182
Open AccessTowards Two-Stage Counterfactual Learning to Rank
- Open Access
- 21 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 737-747
Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank
Counterfactual learning to rank (CLTR) can be risky and, in various circumstances, can produce sub-optimal models that hurt performance when deployed. Safe CLTR was introduced to mitigate these risks when using inverse propensity scoring to correct for position bias. However, the existing safety measure for CLTR is not applicable to state-of-the-art CLTR methods, cannot handle trust bias, and relies on specific assumptions about user behavior. Our contributions are two-fold. First, we generalize the existing safe CLTR approach to make it applicable to state-of-the-art doubly robust CLTR and trust bias. Second, we propose a novel approach, proximal ranking policy optimization (PRPO), that provides safety in deployment without assumptions about user behavior. PRPO removes incentives for learning ranking behavior that is too dissimilar to a safe ranking model. Thereby, PRPO imposes a limit on how much learned models can degrade performance metrics, without relying on any specific user assumptions. Our experiments show that both our novel safe doubly robust method and PRPO provide higher performance than the existing safe inverse propensity scoring approach. However, in unexpected circumstances, the safe doubly robust approach can become unsafe and bring detrimental performance. In contrast, PRPO always maintains safety, even in maximally adversarial situations. By avoiding assumptions, PRPO is the first method with unconditional safety in deployment that translates to robust safety for real-world applications.
- 21 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 737-747
Open AccessPractical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank
Counterfactual learning to rank (CLTR) can be risky and, in various circumstances, can produce sub-optimal models that hurt performance when deployed. Safe CLTR was introduced to mitigate these risks when using inverse propensity scoring to correct for position bias. However, the existing safety measure for CLTR is not applicable to state-of-the-art CLTR methods, cannot handle trust bias, and relies on specific assumptions about user behavior. Our contributions are two-fold. First, we generalize the existing safe CLTR approach to make it applicable to state-of-the-art doubly robust CLTR and trust bias. Second, we propose a novel approach, proximal ranking policy optimization (PRPO), that provides safety in deployment without assumptions about user behavior. PRPO removes incentives for learning ranking behavior that is too dissimilar to a safe ranking model. Thereby, PRPO imposes a limit on how much learned models can degrade performance metrics, without relying on any specific user assumptions. Our experiments show that both our novel safe doubly robust method and PRPO provide higher performance than the existing safe inverse propensity scoring approach. However, in unexpected circumstances, the safe doubly robust approach can become unsafe and bring detrimental performance. In contrast, PRPO always maintains safety, even in maximally adversarial situations. By avoiding assumptions, PRPO is the first method with unconditional safety in deployment that translates to robust safety for real-world applications.
- Open Access
- 8 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 722-732
Optimal Baseline Corrections for Off-Policy Contextual Bandits
The off-policy learning paradigm allows for recommender systems and general ranking applications to be framed as decision-making problems, where we aim to learn decision policies that optimize an unbiased offline estimate of an online reward metric. With unbiasedness comes potentially high variance, and prevalent methods exist to reduce estimation variance. These methods typically make use of control variates, either additive (i.e., baseline corrections or doubly robust methods) or multiplicative (i.e., self-normalisation). Our work unifies these approaches by proposing a single framework built on their equivalence in learning scenarios. The foundation of our framework is the derivation of an equivalent baseline correction for all of the existing control variates. Consequently, our framework enables us to characterize the variance-optimal unbiased estimator and provide a closed-form solution for it. This optimal estimator brings significantly improved performance in both evaluation and learning, and minimizes data requirements. Empirical observations corroborate our theoretical findings.
- 8 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 722-732
Open AccessOptimal Baseline Corrections for Off-Policy Contextual Bandits
The off-policy learning paradigm allows for recommender systems and general ranking applications to be framed as decision-making problems, where we aim to learn decision policies that optimize an unbiased offline estimate of an online reward metric. With unbiasedness comes potentially high variance, and prevalent methods exist to reduce estimation variance. These methods typically make use of control variates, either additive (i.e., baseline corrections or doubly robust methods) or multiplicative (i.e., self-normalisation). Our work unifies these approaches by proposing a single framework built on their equivalence in learning scenarios. The foundation of our framework is the derivation of an equivalent baseline correction for all of the existing control variates. Consequently, our framework enables us to characterize the variance-optimal unbiased estimator and provide a closed-form solution for it. This optimal estimator brings significantly improved performance in both evaluation and learning, and minimizes data requirements. Empirical observations corroborate our theoretical findings.
- Open Access
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 1118-1121
Unbiased Learning to Rank: On Recent Advances and Practical Applications
Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both an introduction to the core concepts of the field and an overview of recent advancements in its foundations, along with several applications of its methods. The tutorial is divided into four parts: Firstly, we give an overview of the different forms of bias that can be addressed with ULTR methods. Secondly, we present a comprehensive discussion of the latest estimation techniques in the ULTR field. Thirdly, we survey published results of ULTR in real-world applications. Fourthly, we discuss the connection between ULTR and fairness in ranking. We end by briefly reflecting on the future of ULTR research and its applications. This tutorial is intended to benefit both researchers and industry practitioners interested in developing new ULTR solutions or utilizing them in real-world applications.
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 1118-1121
Open AccessUnbiased Learning to Rank: On Recent Advances and Practical Applications
Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both an introduction to the core concepts of the field and an overview of recent advancements in its foundations, along with several applications of its methods. The tutorial is divided into four parts: Firstly, we give an overview of the different forms of bias that can be addressed with ULTR methods. Secondly, we present a comprehensive discussion of the latest estimation techniques in the ULTR field. Thirdly, we survey published results of ULTR in real-world applications. Fourthly, we discuss the connection between ULTR and fairness in ranking. We end by briefly reflecting on the future of ULTR research and its applications. This tutorial is intended to benefit both researchers and industry practitioners interested in developing new ULTR solutions or utilizing them in real-world applications.
- Open Access
- 15 December 2023
- Published by Association for Computing Machinery (ACM)
- p. 145-148
Recent Advancements in Unbiased Learning to Rank
- 15 December 2023
- Published by Association for Computing Machinery (ACM)
- p. 145-148
Open AccessRecent Advancements in Unbiased Learning to Rank
- Open Access
- 9 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 87-93
A Deep Generative Recommendation Method for Unbiased Learning from Implicit Feedback
Variational autoencoders (VAEs) are the state-of-the-art model for recommendation with implicit feedback signals. Unfortunately, implicit feedback suffers from selection bias, e.g., popularity bias, position bias, etc., and as a result, training from such signals produces biased recommendation models. Existing methods for debiasing the learning process have not been applied in a generative setting. We address this gap by introducing an inverse propensity scoring (IPS) based method for training VAEs from implicit feedback data in an unbiased way. Our IPS-based estimator for the VAE training objective, VAE-IPS, is provably unbiased w.r.t. selection bias. Our experimental results show that the proposed VAE-IPS model reaches significantly higher performance than existing baselines. Our contributions enable practitioners to combine state-of-the-art VAE recommendation techniques with the advantages of bias mitigation for implicit feedback.
- 9 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 87-93
Open AccessA Deep Generative Recommendation Method for Unbiased Learning from Implicit Feedback
Variational autoencoders (VAEs) are the state-of-the-art model for recommendation with implicit feedback signals. Unfortunately, implicit feedback suffers from selection bias, e.g., popularity bias, position bias, etc., and as a result, training from such signals produces biased recommendation models. Existing methods for debiasing the learning process have not been applied in a generative setting. We address this gap by introducing an inverse propensity scoring (IPS) based method for training VAEs from implicit feedback data in an unbiased way. Our IPS-based estimator for the VAE training objective, VAE-IPS, is provably unbiased w.r.t. selection bias. Our experimental results show that the proposed VAE-IPS model reaches significantly higher performance than existing baselines. Our contributions enable practitioners to combine state-of-the-art VAE recommendation techniques with the advantages of bias mitigation for implicit feedback.
- Open Access
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 249-258
Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 249-258
Open AccessSafe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization
- Open Access
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 3440-3443
Recent Advances in the Foundations and Applications of Unbiased Learning to Rank
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 3440-3443
Open AccessRecent Advances in the Foundations and Applications of Unbiased Learning to Rank
- Open Access
- 16 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 177-181
A Neural Bag-of-Words Point Process Model for User Return Time Prediction in E-commerce
Monitoring and predicting user engagement is essential to gauge the overall health of an E-commerce platform. A healthy active user base indicates that the platform is able to retain users and is performing well on the user satisfaction metric. To measure long-term user satisfaction, predicting the return rate of a user is essential. The frequent return of the user indicates that they are overall satisfied with the platform. To this end, we consider the problem of predicting users’ return time on the platform given their historical interactions. The current state-of-the-art models for user return time prediction are based on recurrent neural network, which models the sequence of user interactions and predicts the return time using a Temporal Point Process based formulation. However, it is well-known that the inference time for these models grows as the sequence length increases, due to the complex recurrent and gating mechanisms, which deems them unfit to be used in a real-time prediction setting. Towards this end, we propose a lightweight and simple neural bag-of-words-based model for user return time prediction, which considers the user activity trail as a bag-of-words embedding model and performs a simple aggregation operation, used for the final prediction. We perform experiments on interaction log data from a major e-commerce company, and our proposed bag-of-words model outperforms the complex recurrence-based neural network by 6.14% and 4.81% on average, in terms of the Root-Mean-Squared-Error and Mean-Absolute-Error, respectively. We also compare the inference time of our method to the recurrent neural network-based method, with an overall reduction of 78.5% in terms of the wall-clock time.
- 16 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 177-181
Open AccessA Neural Bag-of-Words Point Process Model for User Return Time Prediction in E-commerce
Monitoring and predicting user engagement is essential to gauge the overall health of an E-commerce platform. A healthy active user base indicates that the platform is able to retain users and is performing well on the user satisfaction metric. To measure long-term user satisfaction, predicting the return rate of a user is essential. The frequent return of the user indicates that they are overall satisfied with the platform. To this end, we consider the problem of predicting users’ return time on the platform given their historical interactions. The current state-of-the-art models for user return time prediction are based on recurrent neural network, which models the sequence of user interactions and predicts the return time using a Temporal Point Process based formulation. However, it is well-known that the inference time for these models grows as the sequence length increases, due to the complex recurrent and gating mechanisms, which deems them unfit to be used in a real-time prediction setting. Towards this end, we propose a lightweight and simple neural bag-of-words-based model for user return time prediction, which considers the user activity trail as a bag-of-words embedding model and performs a simple aggregation operation, used for the final prediction. We perform experiments on interaction log data from a major e-commerce company, and our proposed bag-of-words model outperforms the complex recurrence-based neural network by 6.14% and 4.81% on average, in terms of the Root-Mean-Squared-Error and Mean-Absolute-Error, respectively. We also compare the inference time of our method to the recurrent neural network-based method, with an overall reduction of 78.5% in terms of the wall-clock time.