Clustering-based Imputation for Dropout Buyers in Large-scale Online Experimentation
Preprint
Abstract
In online experimentation, appropriate metrics (e.g., purchase) provide strong evidence to support hypotheses and enhance the decision-making process. However, incomplete metrics are frequently occurred in the online experimentation, making the available data to be much fewer than the planned online experiments (e.g., A/B testing). In this work, we introduce the concept of dropout buyers and categorize users with incomplete metric values into two groups: visitors and dropout buyers. For the analysis of incomplete metrics, we propose a clustering-based imputation method using $k$-nearest neighbors. Our proposed imputation method considers both the experiment-specific features and users' activities along their shopping paths, allowing different imputation values for different users. To facilitate efficient imputation of large-scale data sets in online experimentation, the proposed method uses a combination of stratification and clustering. The performance of the proposed method is compared to several conventional methods in both simulation studies and a real online experiment at eBay.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Toward Optimal Variance Reduction in Online Controlled ExperimentsTechnometrics, 2022
- Locally Optimal Design for A/B Tests in the Presence of Covariates and Network DependenceTechnometrics, 2022
- Driving safety assessment for ride-hailing driversAccident Analysis & Prevention, 2021
- Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysisBMC Medical Research Methodology, 2020
- What is the difference between missing completely at random and missing at random?International Journal of Epidemiology, 2014
- Statistical Analysis of Randomized Experiments with Non-Ignorable Missing Binary Outcomes: An Application to a Voting ExperimentJournal of the Royal Statistical Society Series C: Applied Statistics, 2008
- Intentions, plans, and the subtle psychology of voter turnout2008
- Estimating the Effect of Unearned Income on Labor Earnings, Savings, and Consumption: Evidence from a Survey of Lottery PlayersAmerican Economic Review, 2001
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987