DISTRIBUTED MINING OF ASSOCIATION RULES BASED ON REDUCING THE SUPPORT THRESHOLD
- 1 December 2008
- journal article
- research article
- Published by World Scientific Pub Co Pte Ltd in International Journal on Artificial Intelligence Tools
- Vol. 17 (06) , 1109-1129
Abstract
One of the most important data mining problems is learning association rules of the form "90% of the customers that purchase product x also purchase product y". Discovering association rules from huge volumes of data requires substantial processing power. In this paper we present an efficient distributed algorithm for mining association rules that reduces the time complexity in a magnitude that renders as suitable for scaling up to very large data sets. The proposed algorithm is based on partitioning the initial data set into subsets and processing each subset in parallel. The proposed algorithm can maintain the set of association rules that are extracted when applying an association rule mining algorithm to all the data, by reducing the support threshold during processing the subsets. The above are confirmed by empirical tests that we present and which also demonstrate the utility of the method.Keywords
This publication has 6 references indexed in Scilit:
- A METHODOLOGY FOR SCALING UP CLASSIFICATION ALGORITHMSInternational Journal on Artificial Intelligence Tools, 2004
- On distributing the clustering processPattern Recognition Letters, 2002
- Algorithms for computing association rules using a partial-support treeKnowledge-Based Systems, 2000
- Effect of Data Distribution in Parallel Mining of AssociationsData Mining and Knowledge Discovery, 1999
- On the Accuracy of Meta-learning for Scalable Data MiningJournal of Intelligent Information Systems, 1997
- Parallel mining of association rulesIEEE Transactions on Knowledge and Data Engineering, 1996