Neural Feature Search: A Neural Architecture for Automated Feature Engineering
Authors:
- 1 November 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 71-80
Abstract
Feature engineering is a crucial step for developing effective machine learning models. Traditionally, feature engineering is performed manually, which requires much domain knowledge and is time-consuming. In recent years, many automated feature engineering methods have been proposed. These methods improve the accuracy of a machine learning model by automatically transforming the original features into a set of new features. However, existing methods either lack ability to perform high-order transformations or suffer from the feature space explosion problem. In this paper, we present Neural Feature Search (NFS), a novel neural architecture for automated feature engineering. We utilize a recurrent neural network based controller to transform each raw feature through a series of transformation functions. The controller is trained through reinforcement learning to maximize the expected performance of the machine learning algorithm. Extensive experiments on public datasets illustrate that our neural architecture is effective and outperforms the existing state-of-the-art automated feature engineering methods. Our architecture can efficiently capture potentially valuable high-order transformations and mitigate the feature explosion problem.Keywords
This publication has 21 references indexed in Scilit:
- Deep feature synthesis: Towards automating data science endeavors2015
- Strengthening learning algorithms by feature discoveryInformation Sciences, 2012
- Programming by optimizationCommunications of the ACM, 2012
- Automated feature generation from structured knowledge2011
- Reinforcement learning of motor skills with policy gradientsNeural Networks, 2008
- Feature Construction and Selection Using Genetic Programming and a Genetic Algorithm2003
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- Q-learningMachine Learning, 1992
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992