Mutation is all you need
Preprint
Abstract
Neural architecture search (NAS) promises to make deep learning accessible to non-experts by automating architecture engineering of deep neural networks. BANANAS is one state-of-the-art NAS method that is embedded within the Bayesian optimization framework. Recent experimental findings have demonstrated the strong performance of BANANAS on the NAS-Bench-101 benchmark being determined by its path encoding and not its choice of surrogate model. We present experimental results suggesting that the performance of BANANAS on the NAS-Bench-301 benchmark is determined by its acquisition function optimizer, which minimally mutates the incumbent.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- mlr3mbo: Flexible Bayesian Optimization2022
- BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture SearchProceedings of the AAAI Conference on Artificial Intelligence, 2021
- bbotk: Black-Box Optimization Toolkit2020
- mlr3: A modern object-oriented machine learning framework in RJournal of Open Source Software, 2019
- mlr3pipelines: Preprocessing Operators and Pipelines for 'mlr3'2019
- reticulate: Interface to 'Python'2017
- ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R2017
- data.table: Extension of `data.frame`2006
- Efficient Global Optimization of Expensive Black-Box FunctionsJournal of Global Optimization, 1998
- On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two SamplesBiometrika, 1933