Statistically Rigorous Testing of Clustering Implementations
Authors:
- 1 April 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 91-98
Abstract
Clustering is a widely-used and well-studied AI branch, but defining clustering correctness, as well as verifying and validating clustering implementations, remains a challenge. To address this, we propose a statistically rigorous approach that couples differential clustering with statistical hypothesis testing, namely we conduct statistical hypothesis testing on the outcome (distribution) of differential clustering to reveal problematic outcomes. We employed this approach on widely-used clustering algorithms implemented in popular ML toolkits; the toolkits were tasked with clustering datasets from the Penn Machine Learning Benchmark. The results indicate that there are statistically significant differences in clustering outcomes in a variety of scenarios where users might not expect clustering outcome variation.Keywords
This publication has 6 references indexed in Scilit:
- SmokeOut: An Approach for Testing Clustering Implementations2019
- K-means properties on six clustering benchmark datasetsApplied Intelligence, 2018
- PMLB: a large benchmark suite for machine learning evaluation and comparisonBioData Mining, 2017
- Stability-Based Validation of Clustering SolutionsNeural Computation, 2004
- Properties of the Hubert-Arable Adjusted Rand Index.Psychological Methods, 2004
- Comparing partitionsJournal of Classification, 1985