James McQueen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
James McQueen has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Doug Hains
Amazon.com Inc
2 shared publications - Rui Song
Amazon, Seattle, USA
2 shared publications - Runzhe Wan
Amazon, Seattle, USA
2 shared publications - Yu Liu
Amazon, Seattle, USA
2 shared publications - Jinxiang Gu
Amazon, Seattle, USA
1 shared publication - Yian Huang
Columbia Univeristy, New York, USA
1 shared publication
Latest Publications
- Open Access
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 374-383
Know When to Fold: Futility-Aware Early Termination in Online Experiments
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 374-383
Open AccessKnow When to Fold: Futility-Aware Early Termination in Online Experiments
- Open Access
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 5016-5027
Experimentation Platforms Meet Reinforcement Learning: Bayesian Sequential Decision-Making for Continuous Monitoring
With the growing needs of online A/B testing to support the innovation in industry, the opportunity cost of running an experiment becomes non-negligible. Therefore, there is an increasing demand for an efficient continuous monitoring service that allows early stopping when appropriate. Classic statistical methods focus on hypothesis testing and are mostly developed for traditional high-stake problems such as clinical trials, while experiments at online service companies typically have very different features and focuses. Motivated by the real needs, in this paper, we introduce a novel framework that we developed in Amazon to maximize customer experience and control opportunity cost. We formulate the problem as a Bayesian optimal sequential decision making problem that has a unified utility function. We discuss extensively practical design choices and considerations. We further introduce how to solve the optimal decision rule via Reinforcement Learning and scale the solution. We show the effectiveness of this novel approach compared with existing methods via a large-scale meta-analysis on experiments in Amazon.
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 5016-5027
Open AccessExperimentation Platforms Meet Reinforcement Learning: Bayesian Sequential Decision-Making for Continuous Monitoring
With the growing needs of online A/B testing to support the innovation in industry, the opportunity cost of running an experiment becomes non-negligible. Therefore, there is an increasing demand for an efficient continuous monitoring service that allows early stopping when appropriate. Classic statistical methods focus on hypothesis testing and are mostly developed for traditional high-stake problems such as clinical trials, while experiments at online service companies typically have very different features and focuses. Motivated by the real needs, in this paper, we introduce a novel framework that we developed in Amazon to maximize customer experience and control opportunity cost. We formulate the problem as a Bayesian optimal sequential decision making problem that has a unified utility function. We discuss extensively practical design choices and considerations. We further introduce how to solve the optimal decision rule via Reinforcement Learning and scale the solution. We show the effectiveness of this novel approach compared with existing methods via a large-scale meta-analysis on experiments in Amazon.