Abstract
In an era of pervasive digitalization, the growing volume and variety of data streams poses a new challenge to the efficient running of data-driven optimization algorithms. Targeting scalable multiobjective evolution under large-instance data, this article proposes the general idea of using subsampled small-data tasks as helpful minions (i.e., auxiliary source tasks) to quickly optimize for large datasets—via an evolutionary multitasking framework. Within this framework, a novel computational resource allocation strategy is designed to enable the effective utilization of the minions while guarding against harmful negative transfers. To this end, an intertask empirical correlation measure is defined and approximated via Bayes’ rule, which is then used to allocate resources online in proportion to the inferred degree of source–target correlation. In the experiments, the performance of the proposed algorithm is verified on: 1) sample average approximations of benchmark multiobjective optimization problems under uncertainty and 2) practical multiobjective hyperparameter tuning of deep neural network models. The results show that the proposed algorithm can obtain up to about 73% speedup relative to existing approaches, demonstrating its ability to efficiently tackle real-world multiobjective optimization involving evaluations on large datasets.
Funding Information
  • Data Science and Artificial Intelligence Research Center (DSAIR), School of Computer Science and Engineering at Nanyang Technological University
  • A*STAR AI3 Seed (C211118016)
  • A*STAR Cyber-Physical Production System (CPPS)—Towards Contextual and Intelligent Response Research Program through RIE2020 IAFPP (A19C1a0018)
  • National Natural Science Foundation of China (62206313)

This publication has 63 references indexed in Scilit: