Evaluating clustering in subspace projections of high dimensional data
Open Access
- 1 August 2009
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 2 (1) , 1270-1281
Abstract
Clustering high dimensional data is an emerging research field.Subspace clusteringorprojected clusteringgroup similar objects in subspaces, i.e. projections, of the full space. In the past decade, several clustering paradigms have been developed in parallel, without thorough evaluation and comparison between these paradigms on a common basis.Conclusive evaluation and comparison is challenged by three major issues. First, there is no ground truth that describes the "true" clusters in real world data. Second, a large variety of evaluation measures have been used that reflect different aspects of the clustering result. Finally, in typical publications authors have limited their analysis to their favored paradigm only, while paying other paradigms little or no attention.In this paper, we take a systematic approach to evaluate the major paradigms in a common framework. We study representative clustering algorithms to characterize the different aspects of each paradigm and give a detailed comparison of their properties. We provide a benchmark set of results on a large variety of real world and synthetic data sets. Using different evaluation measures, we broaden the scope of the experimental analysis and create a common baseline for future developments and comparable evaluations in the field. For repeatability, all implementations, data sets and evaluation measures are available on our website.Keywords
This publication has 20 references indexed in Scilit:
- DensEst: Density Estimation for Data Mining in High Dimensional Spaces2009
- HSM: Heterogeneous Subspace Mining in High Dimensional Data2009
- Morpheus2008
- Finding non-redundant, statistically significant regions in high dimensional data2008
- Pleiades: Subspace Clustering and Evaluation2008
- Density-Connected Subspace Clustering for High-Dimensional Data2004
- A Monte Carlo algorithm for fast projective clustering2002
- Adaptive Grids for Clustering Massive Data Sets2001
- Mining frequent patterns without candidate generation2000
- Fast algorithms for projected clustering1999