Selecting a Selection Procedure
Authors:
- 1 December 2007
- journal article
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Management Science
- Vol. 53 (12) , 1916-1932
Abstract
Selection procedures are used in a variety of applications to select the best of a finite set of alternatives. “Best” is defined with respect to the largest mean, but the mean is inferred with statistical sampling, as in simulation optimization. There are a wide variety of procedures, which begs the question of which selection procedure to select. The main contribution of this paper is to identify, through extensive experimentation, the most effective selection procedures when samples are independent and normally distributed. We also (a) summarize the main structural approaches to deriving selection procedures, (b) formalize new sampling allocations and stopping rules, (c) identify strengths and weaknesses of the procedures, (d) identify some theoretical links between them, and (e) present an innovative empirical test bed with the most extensive numerical comparison of selection procedures to date. The most efficient and easiest to control procedures allocate samples with a Bayesian model for uncertainty about the means and use new adaptive stopping rules proposed here.Keywords
This publication has 21 references indexed in Scilit:
- Simulation Allocation for Determining the Best Design in the Presence of Correlated SamplingINFORMS Journal on Computing, 2007
- Selection Procedures with Frequentist Expected Opportunity Cost BoundsOperations Research, 2005
- Sequential selection procedures: Using sample means to improve efficiencyEuropean Journal of Operational Research, 2005
- Using Ranking and Selection to “Clean Up” after Simulation OptimizationOperations Research, 2003
- A fully sequential procedure for indifference-zone selection in simulationACM Transactions on Modeling and Computer Simulation, 2001
- Selecting a good system: procedures and inferenceIIE Transactions, 2001
- An empirical evaluation of several methods to select the best systemACM Transactions on Modeling and Computer Simulation, 1999
- Convergence properties of ordinal comparison in the simulation of discrete event dynamic systemsJournal of Optimization Theory and Applications, 1996
- A lower bound for the correct subset-selection probability and its application to discrete-event system simulationsIEEE Transactions on Automatic Control, 1996
- Bayesian look ahead one stage sampling allocations for selecting the largest normal meanStatistical Papers, 1994