Improve Single-Point Zeroth-Order Optimization Using High-Pass and Low-Pass Filters
Authors:
- Xin ChenXCXin Chen
- Yujie TangYTYujie Tang
- Na LiNLNa Li
Preprint
Abstract
Single-point zeroth-order optimization (SZO) is useful in solving online black-box optimization and control problems in time-varying environments, as it queries the function value only once at each time step. However, the vanilla SZO method is known to suffer from a large estimation variance and slow convergence, which seriously limits its practical application. In this work, we borrow the idea of high-pass and low-pass filters from extremum seeking control (continuous-time version of SZO) and develop a novel SZO method called HLF-SZO by integrating these filters. It turns out that the high-pass filter coincides with the residual feedback method, and the low-pass filter can be interpreted as the momentum method. As a result, the proposed HLF-SZO achieves a much smaller variance and much faster convergence than the vanilla SZO method and empirically outperforms the residual-feedback SZO method, which is verified via extensive numerical experiments.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Safe Model-Free Optimal Voltage Control via Continuous-Time Zeroth-Order Methods2021
- Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization ApproachIEEE Transactions on Automatic Control, 2021
- Robust hybrid zero-order optimization algorithms with acceleration via averaging in timeAutomatica, 2021
- A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning: Principals, Recent Advances, and ApplicationsIEEE Signal Processing Magazine, 2020
- Stochastic online optimization. Single-point and multi-point non-linear multi-armed bandits. Convex and strongly-convex caseAutomation and Remote Control, 2017
- Random Gradient-Free Minimization of Convex FunctionsFoundations of Computational Mathematics, 2015
- On non-local stability properties of extremum seeking controlAutomatica, 2006
- Real‐Time Optimization by Extremum‐Seeking Control2003
- Performance improvement and limitations in extremum seeking controlSystems & Control Letters, 2000
- On the momentum term in gradient descent learning algorithmsNeural Networks, 1999