StreamDM: Advanced Data Mining in Spark Streaming
Authors:
- Albert BifetABAlbert Bifet
- Silviu ManiuSMSilviu Maniu
- Jianfeng QianJQJianfeng Qian
- Guangjian TianGTGuangjian Tian
- Cheng HeCHCheng He
- Wei FanWFWei Fan
- 1 November 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1608-1611
Abstract
Real-time analytics are becoming increasingly important due to the large amount of data that is being created continuously. Drawing from our experiences at Huawei Noah's Ark Lab, we present and demonstrate here StreamDM, a new open source data mining and machine learning library, designed on top of Spark Streaming, an extension of the core Spark API that enables scalable stream processing of data streams. StreamDM is designed to be easily extended and used, either practitioners, developers, or researchers, and is the first library to contain advanced stream mining algorithms for Spark Streaming.Keywords
This publication has 3 references indexed in Scilit:
- StreamKM++ACM Journal of Experimental Algorithmics, 2012
- A Framework for Clustering Evolving Data Streams2003
- Mining high-speed data streams2000