Parallel leap: large-scale maximal pattern mining in a distributed environment
Authors:
- 1 January 2006
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1 (15219097) , 8 pp.
Abstract
When computationally feasible, mining extremely large databases produces tremendously large numbers of frequent patterns. In many cases, it is impractical to mine those datasets due to their sheer size; not only the extent of the existing patterns, but mainly the magnitude of the search space. Many approaches have been suggested such as sequential mining for maximal patterns or searching for all frequent patterns in parallel. So far, those approaches are still not genuinely effective to mine extremely large datasets. In this work we propose a method that combines both strategies efficiently, i.e. mining in parallel for the set of maximal patterns which, to the best of our knowledge, has never been proposed efficiently before. Using this approach we could mine significantly large datasets; with sizes never reported in the literature before. We are able to effectively discover frequent patterns in a database made of billion transactions using a 32 processors cluster in less than 2 hoursKeywords
This publication has 8 references indexed in Scilit:
- Parallel mining of maximal frequent itemsets from databases2004
- Fast parallel association rule mining without candidacy generation2002
- Mining frequent patterns without candidate generation2000
- Efficiently mining long patterns from databases1998
- Asynchronous parallel algorithm for mining association rules on a shared-memory multi-processors1998
- A fast distributed algorithm for mining association rules1996
- Efficient parallel data mining for association rules1995
- Mining association rules between sets of items in large databases1993