MANAS: Multi-Agent Neural Architecture Search
Preprint
Abstract
The Neural Architecture Search (NAS) problem is typically formulated as a graph search problem where the goal is to learn the optimal operations over edges in order to maximise a graph-level global objective. Due to the large architecture parameter space, efficiency is a key bottleneck preventing NAS from its practical use. In this paper, we address the issue by framing NAS as a multi-agent problem where agents control a subset of the network and coordinate to reach optimal architectures. We provide two distinct lightweight implementations, with reduced memory requirements (1/8th of state-of-the-art), and performances above those of much more computationally expensive methods. Theoretically, we demonstrate vanishing regrets of the form O(sqrt(T)), with T being the total number of rounds. Finally, aware that random search is an, often ignored, effective baseline we perform additional experiments on 3 alternative datasets and 2 network configurations, and achieve favourable results in comparison.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Toward Less Constrained Macro-Neural Architecture SearchIEEE Transactions on Neural Networks and Learning Systems, 2023
- NPENAS: Neural Predictor Guided Evolution for Neural Architecture SearchIEEE Transactions on Neural Networks and Learning Systems, 2022
- Regularized Evolution for Image Classifier Architecture SearchProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Very Deep Convolutional Networks for Text Classification2017
- Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit ProblemsFoundations and Trends in Machine Learning, 2012
- Combinatorial banditsJournal of Computer and System Sciences, 2012
- Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categoriesComputer Vision and Image Understanding, 2007
- The Nonstochastic Multiarmed Bandit ProblemSIAM Journal on Computing, 2002