Data Management in Machine Learning
Authors:
Open Access
- 9 May 2017
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1717-1722
Abstract
Large-scale data analytics using statistical machine learning (ML), popularly called advanced analytics, underpins many modern data-driven applications. The data management community has been working for over a decade on tackling data management-related challenges that arise in ML workloads, and has built several systems for advanced analytics. This tutorial provides a comprehensive review of such systems and analyzes key data management challenges and techniques. We focus on three complementary lines of work: (1) integrating ML algorithms and languages with existing data systems such as RDBMSs, (2) adapting data management-inspired techniques such as query optimization, partitioning, and compression to new systems that target ML workloads, and (3) combining data management and ML ideas to build systems that improve ML lifecycle-related tasks. Finally, we identify key open data management challenges for future research in this important area.Keywords
This publication has 78 references indexed in Scilit:
- A few useful things to know about machine learningCommunications of the ACM, 2012
- Spinning fast iterative data flowsProceedings of the VLDB Endowment, 2012
- Optimizing I/O for big array analyticsProceedings of the VLDB Endowment, 2012
- Distributed GraphLabProceedings of the VLDB Endowment, 2012
- On predictive modeling for optimizing transaction execution in parallel OLTP systemsProceedings of the VLDB Endowment, 2011
- TuffyProceedings of the VLDB Endowment, 2011
- Incrementally maintaining classification using an RDBMSProceedings of the VLDB Endowment, 2011
- The Architecture of SciDB2011
- MAD skillsProceedings of the VLDB Endowment, 2009
- A skip-list approach for efficiently processing forecasting queriesProceedings of the VLDB Endowment, 2008