Barry L. Nelson
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Barry L. Nelson has a total of 56 co-authors. Here are the top co-authors by number of shared publications:
- Eunhye Song
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332
8 shared publications - L. Jeff Hong
School of Management and School of Data Science, Fudan University, Shanghai 200433, China
4 shared publications - Andreas Wächter
Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL 60208 USA
3 shared publications - David J. Eckman
Industrial and Systems Engineering, Texas A&M University
3 shared publications - Jeremy Staum
Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208
3 shared publications - Bruce E. Ankenman
Segal Design Institute, McCormick School of Engineering, Northwestern University, Evanston, IL, USA
2 shared publications
Latest Publications
- Open Access
- 2 April 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Modeling and Computer Simulation
- Vol. 36 (2) , 1-24
Plausible Intervals: Global Inference from Limited Simulation of Structured Problems
We introduce a framework for constructing confidence intervals for the performance of a system as a function of a parameter, decision variable or system state, even when the system is not simulated at the particular parameter, decision variable or state. The proposed methods leverage observations from some other simulated model instances and known functional properties of the performance function being evaluated. The intervals, termed p lausible intervals, deliver a desired coverage probability uniformly over all model instances as the minimum sample size at the simulated model instances increases, and they attain the strongest possible consistency from simulating a finite number of model instances. We illustrate the versatility and effectiveness of plausible intervals through two numerical experiments.
- 2 April 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Modeling and Computer Simulation
- Vol. 36 (2) , 1-24
Open AccessPlausible Intervals: Global Inference from Limited Simulation of Structured Problems
We introduce a framework for constructing confidence intervals for the performance of a system as a function of a parameter, decision variable or system state, even when the system is not simulated at the particular parameter, decision variable or state. The proposed methods leverage observations from some other simulated model instances and known functional properties of the performance function being evaluated. The intervals, termed p lausible intervals, deliver a desired coverage probability uniformly over all model instances as the minimum sample size at the simulated model instances increases, and they attain the strongest possible consistency from simulating a finite number of model instances. We illustrate the versatility and effectiveness of plausible intervals through two numerical experiments.
- 10 January 2026
- Published by Elsevier in European Journal of Operational Research
- Vol. 330 (3) , 850-863
Dice and slice simulation optimization for high-dimensional discrete problems
- 10 January 2026
- Published by Elsevier in European Journal of Operational Research
- Vol. 330 (3) , 850-863
Dice and slice simulation optimization for high-dimensional discrete problems
- 1 September 2025
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Operations Research
- Vol. 73 (5) , 2695-2707
Ranking and Contextual Selection
Context-Sensitive Simulation-Based Decisions When There Is No Time to Simulate Stochastic simulation is a powerful tool for discovering system design decisions that are the best possible (optimal) when averaged over real-world uncertainty. However, in applications such as personalized medicine and web content optimization, even better decisions can be made if they are tailored to specific, contemporaneous covariate information, such as patient health history and user reading habits. Unfortunately, in these and similar applications, there is no time to perform a refined simulation optimization. In “Ranking and Contextual Selection,” Keslin, Nelson, Pagnoncelli, Plumlee, and Rahimian use off-the-shelf simulation optimization methods to create a database of covariates and associated decisions that form a covariate-to-decision classifier and an upper confidence bound on its optimality gap when applied to covariates not in the database. A realistic example of web page assortment optimization is presented using a data set from Yahoo!.
- 1 September 2025
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Operations Research
- Vol. 73 (5) , 2695-2707
Ranking and Contextual Selection
Context-Sensitive Simulation-Based Decisions When There Is No Time to Simulate Stochastic simulation is a powerful tool for discovering system design decisions that are the best possible (optimal) when averaged over real-world uncertainty. However, in applications such as personalized medicine and web content optimization, even better decisions can be made if they are tailored to specific, contemporaneous covariate information, such as patient health history and user reading habits. Unfortunately, in these and similar applications, there is no time to perform a refined simulation optimization. In “Ranking and Contextual Selection,” Keslin, Nelson, Pagnoncelli, Plumlee, and Rahimian use off-the-shelf simulation optimization methods to create a database of covariates and associated decisions that form a covariate-to-decision classifier and an upper confidence bound on its optimality gap when applied to covariates not in the database. A realistic example of web page assortment optimization is presented using a data set from Yahoo!.
- Open Access
- 15 July 2025
- Published by Elsevier in European Journal of Operational Research
- Vol. 330 (3) , 701-714
Fifty years of stochastic simulation: Where we are and where we need to go
- 15 July 2025
- Published by Elsevier in European Journal of Operational Research
- Vol. 330 (3) , 701-714
Open AccessFifty years of stochastic simulation: Where we are and where we need to go
- 5 May 2025
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in INFORMS Journal on Data Science
Stochastic Neighbourhood Components Analysis
Distance metric learning is a fundamental task in data mining and is known to enhance the performance of various distance-based algorithms. In this paper, we consider stochastic training data in which repeated feature vectors can belong to different classes, a scenario in which existing methods of metric learning are known to struggle. This type of data is common in stochastic simulations, where multidimensional, recurrent system states are subject to inherent randomness. Classification models on such high-resolution simulation-generated data play a critical role in real-time decision making across diverse applications. This paper presents and implements a stochastic version of the popular neighbourhood components analysis. We demonstrate its behaviour on stochastic data using simulation models and reveal its advantages when used for nearest neighbour classification. Meanwhile, the assumptions of stochastic labelling and repeated feature vectors extend to data from various domains, suggesting that the method can attain broad impact. For example, beyond its applications to system control and decision making with digital twin simulation, it may enhance the analysis of data from sensor networks, recommender systems, and crowdsourced platforms, where stochasticity and recurring feature patterns are typical. Funding: This work was supported by the Engineering and Physical Sciences Research Council–funded STOR-i Centre for Doctoral Training at Lancaster University [Grant EP/L015692/1]. In addition, Barry L. Nelson’s work was supported by the U.S. National Science Foundation [Grant DMS-1854562]. Data Ethics & Reproducibility Note: The code capsule is available on Code Ocean at https://doi.org/10.24433/CO.0189724.v5 and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2023.0018 ).
- 5 May 2025
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in INFORMS Journal on Data Science
Stochastic Neighbourhood Components Analysis
Distance metric learning is a fundamental task in data mining and is known to enhance the performance of various distance-based algorithms. In this paper, we consider stochastic training data in which repeated feature vectors can belong to different classes, a scenario in which existing methods of metric learning are known to struggle. This type of data is common in stochastic simulations, where multidimensional, recurrent system states are subject to inherent randomness. Classification models on such high-resolution simulation-generated data play a critical role in real-time decision making across diverse applications. This paper presents and implements a stochastic version of the popular neighbourhood components analysis. We demonstrate its behaviour on stochastic data using simulation models and reveal its advantages when used for nearest neighbour classification. Meanwhile, the assumptions of stochastic labelling and repeated feature vectors extend to data from various domains, suggesting that the method can attain broad impact. For example, beyond its applications to system control and decision making with digital twin simulation, it may enhance the analysis of data from sensor networks, recommender systems, and crowdsourced platforms, where stochasticity and recurring feature patterns are typical. Funding: This work was supported by the Engineering and Physical Sciences Research Council–funded STOR-i Centre for Doctoral Training at Lancaster University [Grant EP/L015692/1]. In addition, Barry L. Nelson’s work was supported by the U.S. National Science Foundation [Grant DMS-1854562]. Data Ethics & Reproducibility Note: The code capsule is available on Code Ocean at https://doi.org/10.24433/CO.0189724.v5 and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2023.0018 ).
- 1 January 2024
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Operations Research
- Vol. 72 (1) , 336-354
Parallel Adaptive Survivor Selection
Ranking and selection (R&S) procedures in simulation optimization simulate every feasible solution to provide global statistical error control, often selecting a single solution in finite time that is optimal or near-optimal with high probability. By exploiting parallel computing advancements, large-scale problems with hundreds of thousands and even millions of feasible solutions are suitable for R&S. Naively parallelizing existing R&S methods originally designed for a serial computing setting is generally ineffective, however, as many of these conventional methods uphold family-wise error guarantees that suffer from multiplicity and require pairwise comparisons that present a computational bottleneck. Parallel adaptive survivor selection (PASS) is a new framework specifically designed for large-scale parallel R&S. By comparing systems to an adaptive “standard” that is learned as the algorithm progresses, PASS eliminates inferior solutions with false elimination rate control and with computationally efficient aggregate comparisons rather than pairwise comparisons. PASS satisfies desirable theoretical properties and performs effectively on realistic problems.
- 1 January 2024
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Operations Research
- Vol. 72 (1) , 336-354
Parallel Adaptive Survivor Selection
Ranking and selection (R&S) procedures in simulation optimization simulate every feasible solution to provide global statistical error control, often selecting a single solution in finite time that is optimal or near-optimal with high probability. By exploiting parallel computing advancements, large-scale problems with hundreds of thousands and even millions of feasible solutions are suitable for R&S. Naively parallelizing existing R&S methods originally designed for a serial computing setting is generally ineffective, however, as many of these conventional methods uphold family-wise error guarantees that suffer from multiplicity and require pairwise comparisons that present a computational bottleneck. Parallel adaptive survivor selection (PASS) is a new framework specifically designed for large-scale parallel R&S. By comparing systems to an adaptive “standard” that is learned as the algorithm progresses, PASS eliminates inferior solutions with false elimination rate control and with computationally efficient aggregate comparisons rather than pairwise comparisons. PASS satisfies desirable theoretical properties and performs effectively on realistic problems.
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Modeling and Computer Simulation
- Vol. 33 (4) , 1-28
Using Cache or Credit for Parallel Ranking and Selection
In this article, we focus on ranking and selection procedures that sequentially allocate replications to systems by applying some acquisition function. We propose an acquisition function, called gCEI, which exploits the gradient of the complete expected improvement with respect to the number of replications. We prove that the gCEI procedure, which adopts gCEI as the acquisition function in a serial computing environment, achieves the asymptotically optimal static replication allocation of Glynn and Juneja in the limit under a normality assumption. We also propose two procedures, called caching and credit, that extend any acquisition-function-based procedure in a serial environment into both synchronous and asynchronous parallel environments. While allocating replications to systems, both procedures use persistence forecasts for the unavailable outputs of the currently running replications, but differ in usage of the available outputs. We prove that, under certain assumptions, the caching procedure achieves the same asymptotic allocation as in the serial environment. A similar result holds for the credit procedure using gCEI as the acquisition function. In terms of efficiency and effectiveness, the credit procedure empirically performs as well as the caching procedure, despite not carefully controlling the output history as the caching procedure does, and is faster than the serial version without any number-of-replications penalty due to using persistence forecasts. Both procedures are designed to solve small-to-medium-sized problems on computers with a modest number of processors, such as laptops and desktops as opposed to high-performance clusters, and are superior to state-of-the-art parallel procedures in this setting.
- 26 October 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Modeling and Computer Simulation
- Vol. 33 (4) , 1-28
Open AccessUsing Cache or Credit for Parallel Ranking and Selection
In this article, we focus on ranking and selection procedures that sequentially allocate replications to systems by applying some acquisition function. We propose an acquisition function, called gCEI, which exploits the gradient of the complete expected improvement with respect to the number of replications. We prove that the gCEI procedure, which adopts gCEI as the acquisition function in a serial computing environment, achieves the asymptotically optimal static replication allocation of Glynn and Juneja in the limit under a normality assumption. We also propose two procedures, called caching and credit, that extend any acquisition-function-based procedure in a serial environment into both synchronous and asynchronous parallel environments. While allocating replications to systems, both procedures use persistence forecasts for the unavailable outputs of the currently running replications, but differ in usage of the available outputs. We prove that, under certain assumptions, the caching procedure achieves the same asymptotic allocation as in the serial environment. A similar result holds for the credit procedure using gCEI as the acquisition function. In terms of efficiency and effectiveness, the credit procedure empirically performs as well as the caching procedure, despite not carefully controlling the output history as the caching procedure does, and is faster than the serial version without any number-of-replications penalty due to using persistence forecasts. Both procedures are designed to solve small-to-medium-sized problems on computers with a modest number of processors, such as laptops and desktops as opposed to high-performance clusters, and are superior to state-of-the-art parallel procedures in this setting.
- 1 December 2022
- Published by Springer Nature
- p. 353-379
Foundations of Ranking & Selection for Simulation Optimization
- 1 December 2022
- Published by Springer Nature
- p. 353-379
Foundations of Ranking & Selection for Simulation Optimization
- 1 November 2022
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Operations Research
- Vol. 70 (6) , 3473-3489
Plausible Screening Using Functional Properties for Simulations with Large Solution Spaces
Simulation Solution Screening Using Functional Properties Simulation models today give rise to problems with large numbers of simulated scenarios or solutions—more than can be simulated exhaustively. Fortunately, users of these models may be able to verify or infer properties, such as convexity, of a performance measure of interest when viewed as a function over the space of solutions. In “Plausible Screening Using Functional Properties for Simulations with Large Solution Spaces,” Eckman, Plumlee and Nelson introduce a framework in which such properties are exploited to avoid simulating solutions with unacceptable performances. Their methods solve optimization problems that measure how well the result of a limited simulation experiment agrees with the claim that a solution is acceptable. These methods deliver desirable statistical guarantees of confidence and consistency. Numerical experiments illustrate how functional properties coupled with small simulation experiments can avoid many simulations for simulation-optimization problems.
- 1 November 2022
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Operations Research
- Vol. 70 (6) , 3473-3489
Plausible Screening Using Functional Properties for Simulations with Large Solution Spaces
Simulation Solution Screening Using Functional Properties Simulation models today give rise to problems with large numbers of simulated scenarios or solutions—more than can be simulated exhaustively. Fortunately, users of these models may be able to verify or infer properties, such as convexity, of a performance measure of interest when viewed as a function over the space of solutions. In “Plausible Screening Using Functional Properties for Simulations with Large Solution Spaces,” Eckman, Plumlee and Nelson introduce a framework in which such properties are exploited to avoid simulating solutions with unacceptable performances. Their methods solve optimization problems that measure how well the result of a limited simulation experiment agrees with the claim that a solution is acceptable. These methods deliver desirable statistical guarantees of confidence and consistency. Numerical experiments illustrate how functional properties coupled with small simulation experiments can avoid many simulations for simulation-optimization problems.
- 12 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 30, 1-12
Flat Chance! Using Stochastic Gradient Estimators to Assess Plausible Optimality for Convex Functions
This paper studies methods that identify plausibly near-optimal solutions based on simulation results obtained from only a small subset of feasible solutions. We do so by making use of both noisy estimates of performance and their gradients. Under a convexity assumption on the performance function, these inference methods involve checking only a system of inequalities. We find that these methods can yield more powerful inference at less computational expense compared to methodological predecessors that do not leverage stochastic gradient estimators.
- 12 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 30, 1-12
Flat Chance! Using Stochastic Gradient Estimators to Assess Plausible Optimality for Convex Functions
This paper studies methods that identify plausibly near-optimal solutions based on simulation results obtained from only a small subset of feasible solutions. We do so by making use of both noisy estimates of performance and their gradients. Under a convexity assumption on the performance function, these inference methods involve checking only a system of inequalities. We find that these methods can yield more powerful inference at less computational expense compared to methodological predecessors that do not leverage stochastic gradient estimators.