A Simple Yet Efficient Evolution Strategy for Large-Scale Black-Box Optimization
- 23 October 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 22 (5) , 637-646
Abstract
We propose an evolution strategy algorithm using a sparse plus low rank model for large scale optimization in this paper. We first develop a rank one evolution strategy (R1-ES) using a single principal search direction. It is of linear complexity. Then we extend it to multiple search directions, and develop a rank-m evolution strategy (Rm-ES). We illustrate that the principal search direction accumulates the natural gradients with respect to the distribution mean, and acts as a momentum term. Further, we analyze the optimal low rank approximation to the covariance matrix, and experimentally show that the principal search direction can effectively learn the long valley of the function with predominant search direction. Then we investigate the effects of Hessian on the algorithm performance. We conduct experiments on a class of test problems and the CEC’2010 LSGO benchmarks. The experimental results validate the effectiveness of our proposed algorithms.Keywords
Funding Information
- City University of Hong Kong (A-CityU101/16)
This publication has 26 references indexed in Scilit:
- Impacts of invariance in search: When CMA-ES and PSO face ill-conditioned and non-separable problemsApplied Soft Computing, 2011
- Large scale evolutionary optimization using cooperative coevolutionInformation Sciences, 2008
- Weighted multirecombination evolution strategiesTheoretical Computer Science, 2006
- Fast low-rank modifications of the thin singular value decompositionLinear Algebra and its Applications, 2006
- A Note on the Extended Rosenbrock FunctionEvolutionary Computation, 2006
- Performance Analysis of Evolutionary Optimization With Cumulative Step Length AdaptationIEEE Transactions on Automatic Control, 2004
- Steepest descent with momentum for quadratic functions is a version of the conjugate gradient methodNeural Networks, 2004
- Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES)Evolutionary Computation, 2003
- Completely Derandomized Self-Adaptation in Evolution StrategiesEvolutionary Computation, 2001
- Learning representations by back-propagating errorsNature, 1986