Distributionally Constrained Black-Box Stochastic Gradient Estimation and Optimization
- 1 September 2025
- journal article
- research article
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Operations Research
- Vol. 73 (5) , 2680-2694
Abstract
Constructing stochastic gradient estimators and descent algorithms is a core problem in stochastic optimization. In the black-box setting, namely when only noisy function evaluations are available, such gradient estimators are typically constructed via finite-differencing. In this regard, there have been schemes that aim to obtain gradient estimators for potentially many dimensions simultaneously via only few sample observations or simulation runs. However, for problems with probability simplex constraints, which arise in a range of applications from distributionally robust analysis to inverse model calibration, these schemes run into challenges one way or another when attempting to balance bias-variance in a constraint-compatible manner. This motivates us to create a new design framework for random perturbation generators and estimation schemes that bypass these challenges. Our culminated class of estimators, which is based on the Dirichlet mixtures, is demonstrably effective in distributionally constrained gradient estimation and optimization under various black-box settings.Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- Input Uncertainty in Stochastic Simulation2022
- Model calibration via distributionally robust optimization: On the NASA Langley Uncertainty Quantification ChallengeMechanical Systems and Signal Processing, 2022
- A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free OptimizationFoundations of Computational Mathematics, 2021
- Stochastic simulation under input uncertainty: A ReviewOperations Research Perspectives, 2020
- Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulationsMathematical Programming, 2017
- Robust sample average approximationMathematical Programming, 2017
- Random Gradient-Free Minimization of Convex FunctionsFoundations of Computational Mathematics, 2015
- Generalized Gauss inequalities via semidefinite programmingMathematical Programming, 2015
- Stochastic Simulation: Algorithms and Analysis2007
- Infinitesimal and finite perturbation analysis for queueing networksAutomatica, 1983