GRADIENT-BASED STOCHASTIC OPTIMIZATION METHODS IN BAYESIAN EXPERIMENTAL DESIGN
Top Cited Papers
Open Access
- 1 January 2014
- journal article
- research article
- Published by Begell House Inc. Publishers in International Journal for Uncertainty Quantification
- Vol. 4 (6) , 479-510
Abstract
Optimal experimental design (OED) seeks experiments expected to yield the most useful data for some purpose. In practical circumstances where experiments are time-consuming or resource-intensive, OED can yield enormous savings. We pursue OED for nonlinear systems from a Bayesian perspective, with the goal of choosing experiments that are optimal for parameter inference. Our objective in this context is the expected information gain in model parameters, which in general can only be estimated using Monte Carlo methods. Maximizing this objective thus becomes a stochastic optimization problem. This paper develops gradient-based stochastic optimization methods for the design of experiments on a continuous parameter space. Given a Monte Carlo estimator of expected information gain, we use infinitesimal perturbation analysis to derive gradients of this estimator.We are then able to formulate two gradient-based stochastic optimization approaches: (i) Robbins-Monro stochastic approximation, and (ii) sample average approximation combined with a deterministic quasi-Newton method. A polynomial chaos approximation of the forward model accelerates objective and gradient evaluations in both cases.We discuss the implementation of these optimization methods, then conduct an empirical comparison of their performance. To demonstrate design in a nonlinear setting with partial differential equation forward models, we use the problem of sensor placement for source inversion. Numerical results yield useful guidelines on the choice of algorithm and sample sizes, assess the impact of estimator bias, and quantify tradeoffs of computational cost versus solution quality and robustness.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Simulation-based optimal Bayesian experimental design for nonlinear systemsJournal of Computational Physics, 2013
- Sparse pseudospectral approximation methodComputer Methods in Applied Mechanics and Engineering, 2012
- Bayesian experimental design for the active nitridation of graphite by atomic nitrogenExperimental Thermal and Fluid Science, 2012
- Supply chain design under uncertainty using sample average approximation and dual decompositionEuropean Journal of Operational Research, 2009
- Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problemsJournal of Computational Physics, 2009
- Robust Stochastic Approximation Approach to Stochastic ProgrammingSIAM Journal on Optimization, 2009
- Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input SpaceSIAM Journal on Scientific Computing, 2008
- Stochastic spectral methods for efficient Bayesian solution of inverse problemsJournal of Computational Physics, 2007
- A new stochastic approach to transient heat conduction modeling with uncertaintyInternational Journal of Heat and Mass Transfer, 2003
- Uncertainty quantification in reacting-flow simulations through non-intrusive spectral projectionCombustion and Flame, 2003