Mining flipping correlations from large datasets with taxonomies
Authors:
Open Access
- 1 December 2011
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 5 (4) , 370-381
Abstract
In this paper we introduce a new type of pattern -- a flipping correlation pattern. The flipping patterns are obtained from contrasting the correlations between items at different levels of abstraction. They represent surprising correlations, both positive and negative, which are specific for a given abstraction level, and which "flip" from positive to negative and vice versa when items are generalized to a higher level of abstraction. We design an efficient algorithm for finding flipping correlations, the Flipper algorithm, which outperforms naïve pattern mining methods by several orders of magnitude. We apply Flipper to real-life datasets and show that the discovered patterns are non-redundant, surprising and actionable. Flipper finds strong contrasting correlations in itemsets with low-to-medium support, while existing techniques cannot handle the pattern discovery in this frequency range.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Re-examination of interestingness measures in pattern mining: a unified frameworkData Mining and Knowledge Discovery, 2010
- Implications of Probabilistic Data Modeling for Mining Association Rules2006
- Efficient mining of both positive and negative association rulesACM Transactions on Information Systems, 2004
- Mining Positive and Negative Association Rules: An Approach for Confined Rules2004
- Selecting the right interestingness measure for association patterns2002
- Analysis of matched mRNA measurements from two different microarray technologiesBioinformatics, 2002
- Mining frequent patterns without candidate generation2000
- Hierarchy-based mining of association rules in data warehouses2000
- Pruning and summarizing the discovered associations1999
- Mining association rules between sets of items in large databases1993