Amit Gandhi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Amit Gandhi has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Alex Deng
Seattle Airbnb San Francisco Airbnb San Francisco Airbnb San Francisco WA CA CA CA USA
1 shared publication - Lo-Hua Yuan
Seattle Airbnb San Francisco Airbnb San Francisco Airbnb San Francisco WA CA CA CA USA
1 shared publication - Luke Hagar
Clinical Trials Capability, The University of Queensland
1 shared publication - Nathaniel Tyler Stevens
Department of Statistics and Actuarial Science, University of Waterloo
1 shared publication - Tatiana Xifara
University of Waterloo
1 shared publication
Latest Publications
- Open Access
From Augmentation to Decomposition: A New Look at CUPED in 2023
Ten years ago, CUPED (Controlled Experiments Utilizing Pre-Experiment Data) mainstreamed the idea of variance reduction leveraging pre-experiment covariates. Since its introduction, it has been implemented, extended, and modernized by major online experimentation platforms. Many researchers and practitioners often interpret CUPED as a regression adjustment. In this article, we clarify its similarities and differences to regression adjustment and present CUPED as a more general augmentation framework which is closer to the spirit of the 2013 paper. We show that the augmentation view naturally leads to cleaner developments of variance reduction beyond simple average metrics, including ratio metrics and percentile metrics. Moreover, the augmentation view can go beyond using pre-experiment data and leverage in-experiment data, leading to significantly larger variance reduction. We further introduce metric decomposition using approximate null augmentation (ANA) as a mental model for in-experiment variance reduction. We study it under both a Bayesian framework and a frequentist optimal proxy metric framework. Metric decomposition arises naturally in conversion funnels, so this work has broad applicability.
Open AccessFrom Augmentation to Decomposition: A New Look at CUPED in 2023
Ten years ago, CUPED (Controlled Experiments Utilizing Pre-Experiment Data) mainstreamed the idea of variance reduction leveraging pre-experiment covariates. Since its introduction, it has been implemented, extended, and modernized by major online experimentation platforms. Many researchers and practitioners often interpret CUPED as a regression adjustment. In this article, we clarify its similarities and differences to regression adjustment and present CUPED as a more general augmentation framework which is closer to the spirit of the 2013 paper. We show that the augmentation view naturally leads to cleaner developments of variance reduction beyond simple average metrics, including ratio metrics and percentile metrics. Moreover, the augmentation view can go beyond using pre-experiment data and leverage in-experiment data, leading to significantly larger variance reduction. We further introduce metric decomposition using approximate null augmentation (ANA) as a mental model for in-experiment variance reduction. We study it under both a Bayesian framework and a frequentist optimal proxy metric framework. Metric decomposition arises naturally in conversion funnels, so this work has broad applicability.