Data Farming the Parameters of Simulation-Optimization Solvers
Open Access
- 13 August 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Modeling and Computer Simulation
- Vol. 34 (4) , 1-29
Abstract
The performance of a simulation-optimization algorithm, a.k.a. a solver, depends on its parameter settings. Much of the research to date has focused on how a solver’s parameters affect its convergence and other asymptotic behavior. While these results are important for providing a theoretical understanding of a solver, they can be of limited utility to a user who must set up and run the solver on a particular problem. When running a solver in practice, good finite-time performance is paramount. In this article, we explore the relationship between a solver’s parameter settings and its finite-time performance by adopting a data farming approach. The approach involves conducting and analyzing the outputs of a designed experiment wherein the factors are the solver’s parameters and the responses are assorted performance metrics measuring the solver’s speed and solution quality over time. We demonstrate this approach with a study of the ASTRO-DF solver when solving a stochastic activity network problem and an inventory control problem. Through these examples, we show that how some of the solver’s parameters are set greatly affects its ability to achieve rapid, reliable progress and gain insights into the solver’s inner workings. We discuss the implications of using this framework for tuning solver parameters, as well as for addressing related questions of interest to solver specialists and generalists.Keywords
Funding Information
- National Science Foundation (CMMI-2226347 and CMMI-2206972)
This publication has 36 references indexed in Scilit:
- Automated machine learning: Review of the state-of-the-art and opportunities for healthcareArtificial Intelligence in Medicine, 2020
- Derivative-free optimization methodsActa Numerica, 2019
- Orthogonal second‐order space‐filling designs with insights from simulation experiments to support test planningQuality and Reliability Engineering International, 2018
- Comparison of Gaussian process modeling softwareEuropean Journal of Operational Research, 2018
- Efficient, nearly orthogonal-and-balanced, mixed designs: an effective way to conduct trade-off analyses via simulationJournal of Simulation, 2013
- Continuous optimization algorithms for tuning real and integer parameters of swarm intelligence algorithmsSwarm Intelligence, 2011
- Sequential Model-Based Optimization for General Algorithm Configuration2011
- Fine-Tuning Algorithm Parameters Using the Design of Experiments Approach2011
- Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-ModelsJournal of Global Optimization, 2006
- Monte Carlo bounding techniques for determining solution quality in stochastic programsOperations Research Letters, 1999