Angel: a new large-scale machine learning system
Authors:
Open Access
- 24 February 2017
- journal article
- research article
- Published by Oxford University Press (OUP) in National Science Review
- Vol. 5 (2) , 216-236
Abstract
Machine Learning (ML) techniques now are ubiquitous tools to extract structural information from data collections. With the increasing volume of data, large-scale ML applications require an efficient implementation to accelerate the performance. Existing systems parallelize algorithms through either data parallelism or model parallelism. But data parallelism cannot obtain good statistical efficiency due to the conflicting updates to parameters while the performance is damaged by global barriers in model parallel methods. In this paper, we propose a new system, named Angel, to facilitate the development of large-scale ML applications in production environment. By allowing concurrent updates to model across different groups and scheduling the updates in each group, Angel can achieve a good balance between hardware efficiency and statistical efficiency. Besides, Angel reduces the network latency by overlapping the parameter pulling and update computing and also utilizes the sparseness of data to avoid the pulling of unnecessary parameters. We also enhance the usability of Angel by providing a set of efficient tools to integrate with application pipelines and provisioning efficient fault tolerance mechanisms. We conduct extensive experiments to demonstrate the superiority of Angel.Keywords
Funding Information
- National Natural Science Foundation of China (61572039)
- National Basic Research Program of China (2014CB340405)
- Shenzhen Government Research Project (JCYJ20151014093505032)
This publication has 10 references indexed in Scilit:
- Big Graph Analyses: From Queries to Dependencies and Association RulesData Science and Engineering, 2017
- PeacockACM Transactions on Intelligent Systems and Technology, 2015
- Petuum: A New Platform for Distributed Machine Learning on Big DataIEEE Transactions on Big Data, 2015
- DimmWittedProceedings of the VLDB Endowment, 2014
- PAGE: A Partition Aware Engine for Parallel Graph ComputationIEEE Transactions on Knowledge and Data Engineering, 2014
- Coordinate descent algorithms for lasso penalized regressionThe Annals of Applied Statistics, 2008
- MapReduceCommunications of the ACM, 2008
- Finding scientific topicsProceedings of the National Academy of Sciences, 2004
- A bridging model for parallel computationCommunications of the ACM, 1990
- Use of Dummy Variables in Regression EquationsJournal of the American Statistical Association, 1957