Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning
Preprint
Abstract
Existing Bayesian Optimization (BO) methods typically balance exploration and exploitation to optimize costly objective functions. However, these methods often suffer from a significant one-step bias, which may lead to convergence towards local optima and poor performance in complex or high-dimensional tasks. Recently, Black-Box Optimization (BBO) has achieved success across various scientific and engineering domains, particularly when function evaluations are costly and gradients are unavailable. Motivated by this, we propose the Reinforced Energy-Based Model for Bayesian Optimization (REBMBO), which integrates Gaussian Processes (GP) for local guidance with an Energy-Based Model (EBM) to capture global structural information. Notably, we define each Bayesian Optimization iteration as a Markov Decision Process (MDP) and use Proximal Policy Optimization (PPO) for adaptive multi-step lookahead, dynamically adjusting the depth and direction of exploration to effectively overcome the limitations of traditional BO methods. We conduct extensive experiments on synthetic and real-world benchmarks, confirming the superior performance of REBMBO. Additional analyses across various GP configurations further highlight its adaptability and robustness.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Recent Advances in Bayesian OptimizationACM Computing Surveys, 2023
- NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and SizeIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- Two decades of blackbox optimization applicationsEURO Journal on Computational Optimization, 2021
- A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT2020
- Simple Yet Effective Fine-Tuning of Deep CNNs Using an Auxiliary Classification Loss for Remote Sensing Scene ClassificationRemote Sensing, 2019
- Flex ddG: Rosetta Ensemble-Based Estimation of Changes in Protein–Protein Binding Affinity upon MutationThe Journal of Physical Chemistry B, 2018
- Automatic Chemical Design Using a Data-Driven Continuous Representation of MoleculesACS Central Science, 2018
- Macromolecular Modeling with RosettaAnnual Review of Biochemistry, 2008
- ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problemsIEEE Transactions on Evolutionary Computation, 2006
- Training Products of Experts by Minimizing Contrastive DivergenceNeural Computation, 2002