Deep Crossing
Authors:
Open Access
- 13 August 2016
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 255-262
Abstract
Manually crafted combinatorial features have been the \"secret sauce\" behind many successful models. For web-scale applications, however, the variety and volume of features make these manually crafted features expensive to create, maintain, and deploy. This paper proposes the Deep Crossing model which is a deep neural network that automatically combines features to produce superior models. The input of Deep Crossing is a set of individual features that can be either dense or sparse. The important crossing features are discovered implicitly by the networks, which are comprised of an embedding and stacking layer, as well as a cascade of Residual Units. Deep Crossing is implemented with a modeling tool called the Computational Network Tool Kit (CNTK), powered by a multi-GPU platform. It was able to build, from scratch, two web-scale models for a major paid search engine, and achieve superior results with only a sub-set of the features used in the production models. This demonstrates the potential of using Deep Crossing as a general modeling paradigm to improve existing products, as well as to speed up the development of new models with a fraction of the investment in feature engineering and acquisition of deep domain knowledge.Keywords
This publication has 10 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering2016
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Learning semantic representations using convolutional neural networks for web search2014
- Learning deep structured semantic models for web search using clickthrough data2013
- Machine Learning Paradigms for Speech Recognition: An OverviewIEEE Transactions on Audio, Speech, and Language Processing, 2013
- Factorization Machines2010
- Internet Advertising and the Generalized Second Price Auction: Selling Billions of Dollars Worth of Keywords2005
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in positionBiological Cybernetics, 1980