Cautionary Remarks on the Use of Clusterwise Regression
Authors:
- 19 March 2008
- journal article
- research article
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 43 (1) , 29-49
Abstract
Clusterwise linear regression is a multivariate statistical procedure that attempts to cluster objects with the objective of minimizing the sum of the error sums of squares for the within-cluster regression models. In this article, we show that the minimization of this criterion makes no effort to distinguish the error explained by the within-cluster regression models from the error explained by the clustering process. In some cases, most of the variation in the response variable is explained by clustering the objects, with little additional benefit provided by the within-cluster regression models. Accordingly, there is tremendous potential for overfitting with clusterwise regression, which is demonstrated with numerical examples and simulation experiments. To guard against the misuse of clusterwise regression, we recommend a benchmarking procedure that compares the results for the observed empirical data with those obtained across a set of random permutations of the response measures. We also demonstrate the potential for overfitting via an empirical application related to the prediction of reflective judgment using high school and college performance measures.Keywords
This publication has 39 references indexed in Scilit:
- Reflective Judgment: Theory and Research on the Development of Epistemic Assumptions Through AdulthoodEducational Psychologist, 2004
- Local Optima in K-Means Clustering: What You Don't Know May Hurt You.Psychological Methods, 2003
- Heterogeneity among peer-rejected boys across middle childhood: Developmental pathways of social behavior.Developmental Psychology, 2002
- GLIMMIX: Software for Estimating Mixtures and Mixtures of Generalized Linear ModelsJournal of Classification, 2001
- A bio-mimetic approach to marketing segmentation: Principles and comparative analysisEuropean Journal of Economic and Social Systems, 2000
- A mixture likelihood approach for generalized linear modelsJournal of Classification, 1995
- Consumer benefit segmentation using clusterwise linear regressionInternational Journal of Research in Marketing, 1989
- A maximum likelihood methodology for clusterwise linear regressionJournal of Classification, 1988
- Comparing partitionsJournal of Classification, 1985
- Relation of goal level to performance with a short work period and multiple goal levels.Journal of Applied Psychology, 1982