Deep Learning at Scale and at Ease
Authors:
Open Access
- 2 November 2016
- journal article
- other
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 12 (4s) , 1-25
Abstract
Recently, deep learning techniques have enjoyed success in various multimedia applications, such as image classification and multimodal data analysis. Large deep learning models are developed for learning rich representations of complex data. There are two challenges to overcome before deep learning can be widely adopted in multimedia and other applications. One is usability, namely the implementation of different models and training algorithms must be done by nonexperts without much effort, especially when the model is large and complex. The other is scalability, namely the deep learning system must be able to provision for a huge demand of computing resources for training large models with massive datasets. To address these two challenges, in this article we design a distributed deep learning platform called SINGA , which has an intuitive programming model based on the common layer abstraction of deep learning models. Good scalability is achieved through flexible distributed training architecture and specific optimization techniques. SINGA runs on both GPUs and CPUs, and we show that it outperforms many other state-of-the-art deep learning systems. Our experience with developing and training deep learning models for real-life multimedia applications in SINGA shows that the platform is both usable and scalable.Keywords
All Related Versions
Funding Information
- A*Star (1321202073)
- National Natural Science Foundation of China (61472348)
- National Research Foundation, Prime Minister's Office, Singapore under its Competitive Research Programme (CRP Award No. NRF-CRP8-2011-08)
- National Research Foundation, Energy Innovation Programme Office, Singapore under Energy Innovation Research Programme (EIRP, Award No. NRF2014EWTEIRP002-026)
This publication has 21 references indexed in Scilit:
- In-memory DatabasesACM SIGMOD Record, 2015
- Effective deep learning-based multi-modal retrievalThe VLDB Journal, 2015
- DimmWittedProceedings of the VLDB Endowment, 2014
- Effective multi-modal retrieval based on stacked auto-encodersProceedings of the VLDB Endowment, 2014
- epiCProceedings of the VLDB Endowment, 2014
- Deep, Big, Simple Neural Nets for Handwritten Digit RecognitionNeural Computation, 2010
- NUS-WIDE2009
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- Giving meanings to WWW images2000
- Efficient BackProp1998