A Data Science Solution for Mining Interesting Patterns from Uncertain Big Data
Authors:
- Carson Kai-Sang LeungCarson Kai-Sang Leung
- Fan JiangFJFan Jiang
- 1 December 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 235-242
Abstract
Nowadays, high volumes of valuable uncertain data can be easily collected or generated at high velocity in many real-life applications. Mining these uncertain Big data is computationally intensive due to the presence of existential probability values associated with items in every transaction in the uncertain data. Each existential probability value expresses the likelihood of that item to be present in a particular transaction in the Big data. In some situations, users may be interested in mining all frequent patterns from these uncertain Big data, in other situations, users may be interested in only a tiny portion of these mined patterns. To reduce the computation and to focus the mining for the latter situations, we propose a tree-based algorithm that (i) allows users to express the patterns to be mined according to their intention via the use of constraints and (ii) uses MapReduce to mine uncertain Big data for only those frequent patterns that satisfy user-specified constraints. Experimental results show the effectiveness of our algorithm in mining interesting patterns from uncertain Big data.Keywords
This publication has 24 references indexed in Scilit:
- Uncertain Frequent Pattern Mining2014
- Mining constrained frequent itemsets from distributed uncertain dataFuture Generation Computer Systems, 2014
- Tightening Upper Bounds to the Expected Support for Uncertain Frequent Pattern MiningProcedia Computer Science, 2014
- HazyCommunications of the ACM, 2013
- Mining Frequent Patterns from Uncertain Data with MapReduce for Big Data Analytics2013
- Mining frequent itemsets over uncertain databasesProceedings of the VLDB Endowment, 2012
- Frequent Itemset Mining with Constraints2009
- MapReduceCommunications of the ACM, 2008
- Efficient dynamic mining of constrained frequent setsACM Transactions on Database Systems, 2003
- Parallel and distributed association mining: a surveyIEEE Concurrency, 1999