Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware
Authors:
Preprint
Abstract
Microarray gene expression has widely attracted the eyes of the public as an efficient tool for cancer diagnosis and classification. However, the very-high dimensionality and the small number of samples make it difficult for traditional machine learning algorithms to address this problem due to the high amount of computations required and overfitting. So far, the existing approaches of processing microarray datasets are still far from satisfactory and they employ two phases, feature selection (or extraction) followed by a machine learning algorithm. In this paper, we show that MultiLayer Perceptrons (MLPs) with adaptive sparse connectivity can directly handle this problem without features selection. Tested on four datasets, our novel results demonstrate that deep learning methods can be applied directly also to high dimensional non-grid like data, while learning from a small amount of labeled examples with imbalanced classes and achieving better accuracy than the traditional two phases approach. Moreover, we have been able to create sparse MLP models with over one million neurons and to train them on a typical laptop without GPU. This is with two orders of magnitude more than the largest MLPs which can run currently on commodity hardware.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network scienceNature Communications, 2018
- A topological insight into restricted Boltzmann machinesMachine Learning, 2016
- Understanding brain networks and brain organizationPhysics of Life Reviews, 2014
- Fast wrapper feature subset selection in high-dimensional datasets by means of filter re-rankingKnowledge-Based Systems, 2012
- Distributed computation of the knn graph for large high-dimensional point setsJournal of Parallel and Distributed Computing, 2007
- Gene Expression Profiling of Gliomas Strongly Predicts SurvivalCancer Research, 2004
- Pitfalls in the Use of DNA Microarray Data for Diagnostic and Prognostic ClassificationJNCI: Journal of the National Cancer Institute, 2003
- Exploring complex networksNature, 2001
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996
- Learning representations by back-propagating errorsNature, 1986