Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation
Authors:
Top Cited Papers
- 1 October 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1294-1303
Abstract
Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search. However, these approaches often report lower accuracy in evaluating the searched architecture or transferring it to another dataset. This is arguably due to the large gap between the architecture depths in search and evaluation scenarios. In this paper, we present an efficient algorithm which allows the depth of searched architectures to grow gradually during the training procedure. This brings two issues, namely, heavier computational overheads and weaker search stability, which we solve using search space approximation and regularization, respectively. With a significantly reduced search time (~7 hours on a single GPU), our approach achieves state-of-the-art performance on both the proxy dataset (CIFAR10 or CIFAR100) and the target dataset (ImageNet). Code is available at https://github.com/chenxin061/pdartsKeywords
This publication has 13 references indexed in Scilit:
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices2018
- Genetic CNN2017
- A genetic programming approach to designing convolutional neural network architectures2017
- Densely Connected Convolutional Networks2017
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- Going deeper with convolutions2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- ImageNet: A large-scale hierarchical image database2009
- Evolving Neural Networks through Augmenting TopologiesEvolutionary Computation, 2002