Controlling Sequential Hybrid Evolutionary Algorithm by Q-Learning [Research Frontier] [Research Frontier]
- 2 February 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computational Intelligence Magazine
- Vol. 18 (1) , 84-103
Abstract
Many state-of-the-art evolutionary algorithms (EAs) can be categorized as sequential hybrid EAs, in which various EAs are sequentially executed. The timing to switch from one EA to another is critical to the performance of the hybrid EA because the switching time determines the allocation of computational resources and thereby it helps balance exploration and exploitation. In this article, a framework for adaptive parameter control for hybrid EAs is proposed, in which the switching time is controlled by a learned agent rather than a manually designed scheme. First the framework is applied to an adaptive differential evolution algorithm, LSHADE, to control when to use the scheme to reduce the population. Then the framework is applied to the algorithm that won the CEC 2018 competition, i.e., the hybrid sampling evolution strategy (HSES), to control when to switch from the univariate sampling phase to the Covariance Matrix Adaptation Evolution Strategy phase. The agents for parameter control in LSHADE and HSES are trained by using Q-learning and deep Q-learning to obtain the learned algorithms Q-LSHADE and DQ-HSES. The results of experiments on the CEC 2014 and 2018 test suites show that the learned algorithms significantly outperform their counterparts and some state-of-the-art EAs.Keywords
Funding Information
- National Natural Science Foundation of China (11991023, 62076197, 61903294)
- National Natural Science Foundation of China (U1811461)
- National Natural Science Foundation of China (11690011)
- National Natural Science Foundation of China (61721002)
This publication has 41 references indexed in Scilit:
- Adaptive population tuning scheme for differential evolutionInformation Sciences, 2013
- HypE: An Algorithm for Fast Hypervolume-Based Many-Objective OptimizationEvolutionary Computation, 2011
- Sequential Model-Based Optimization for General Algorithm Configuration2011
- Differential Evolution with Fitness Diversity Self-adaptation2009
- A modified hybrid EP–SQP approach for dynamic dispatch with valve-point effectInternational Journal of Electrical Power & Energy Systems, 2005
- Exploring dynamic self-adaptive populations in differential evolutionSoft Computing, 2005
- Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES)Evolutionary Computation, 2003
- A hybrid EP and SQP for dynamic economic dispatch with nonsmooth fuel cost functionIEEE Transactions on Power Systems, 2002
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Long Short-Term MemoryNeural Computation, 1997