Using analysis of covariance (ANCOVA) with fallible covariates.
- 1 January 2011
- journal article
- research article
- Published by American Psychological Association (APA) in Psychological Methods
- Vol. 16 (2) , 166-178
Abstract
Analysis of covariance (ANCOVA) is used widely in psychological research implementing nonexperimental designs. However, when covariates are fallible (i.e., measured with error), which is the norm, researchers must choose from among 3 inadequate courses of action: (a) know that the assumption that covariates are perfectly reliable is violated but use ANCOVA anyway (and, most likely, report misleading results); (b) attempt to employ 1 of several measurement error models with the understanding that no research has examined their relative performance and with the added practical difficulty that several of these models are not available in commonly used statistical software; or (c) not use ANCOVA at all. First, we discuss analytic evidence to explain why using ANCOVA with fallible covariates produces bias and a systematic inflation of Type I error rates that may lead to the incorrect conclusion that treatment effects exist. Second, to provide a solution for this problem, we conduct 2 Monte Carlo studies to compare 4 existing approaches for adjusting treatment effects in the presence of covariate measurement error: errors-in-variables (EIV; Warren, White, & Fuller, 1974), Lord's (1960) method, Raaijmakers and Pieters's (1987) method (R&P), and structural equation modeling methods proposed by Sorbom (1978) and Hayduk (1996). Results show that EIV models are superior in terms of parameter accuracy, statistical power, and keeping Type I error close to the nominal value. Finally, we offer a program written in R that performs all needed computations for implementing EIV models so that ANCOVA can be used to obtain accurate results even when covariates are measured with error.Keywords
This publication has 6 references indexed in Scilit:
- Scale Coarseness as a Methodological ArtifactOrganizational Research Methods, 2008
- The Neglected Science and Art of Quasi-ExperimentationOrganizational Research Methods, 2008
- Average causal effects from nonrandomized studies: A practical guide and simulated example.Psychological Methods, 2008
- The Multifaceted Nature of Measurement Artifacts and Its Implications for Estimating Construct-Level RelationshipsOrganizational Research Methods, 2007
- The Role of Measurement Error in Familiar StatisticsOrganizational Research Methods, 2006
- Stereotype Threat Research and the Assumptions Underlying Analysis of Covariance.American Psychologist, 2005