An experimental evaluation of large scale GBDT systems
Authors:
Open Access
- 1 July 2019
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 12 (11) , 1357-1370
Abstract
Gradient boosting decision tree (GBDT) is a widely-used machine learning algorithm in both data analytic competitions and real-world industrial applications. Further, driven by the rapid increase in data volume, efforts have been made to train GBDT in a distributed setting to support large-scale workloads. However, we find it surprising that the existing systems manage the training dataset in different ways, but none of them have studied the impact of data management. To that end, this paper aims to study the pros and cons of different data management methods regarding the performance of distributed GBDT. We first introduce a quadrant categorization of data management policies based on data partitioning and data storage. Then we conduct an in-depth systematic analysis and summarize the advantageous scenarios of the quadrants. Based on the analysis, we further propose a novel distributed GBDT system named Vero, which adopts the unexplored composition of vertical partitioning and row-store and suits for many large-scale cases. To validate our analysis empirically, we implement different quadrants in the same code base and compare them under extensive workloads, and finally compare Vero with other state-of-the-art systems over a wide range of datasets. Our theoretical and experimental results provide a guideline on choosing a proper data management policy for a given workload.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Deep Neural Decision Forests2015
- Practical Lessons from Predicting Clicks on Ads at Facebook2014
- DimmWittedProceedings of the VLDB Endowment, 2014
- Neural Decision Forests for Semantic Image Labelling2014
- A predictive dynamic load balancing algorithm with service differentiation2013
- Further results on the margin explanation of boosting: new algorithm and experimentsScience China Information Sciences, 2012
- Optimization of Collective Communication Operations in MPICHThe International Journal of High Performance Computing Applications, 2005
- Integrating vertical and horizontal partitioning into automated physical database design2004
- Space-efficient online computation of quantile summariesACM SIGMOD Record, 2001
- A Decision-Theoretic Generalization of On-Line Learning and an Application to BoostingJournal of Computer and System Sciences, 1997