Samaradasa Weerahandi

This article presents two approaches to extend classical MANOVA tests to the unequal covariances case. The first approach is illustrated by extending the classical Wilks test, which is valid only when covariances are equal. Such tests will be based on exact probabilities of certain extreme regions. We will also show how tests numerically equivalent to the parametric bootstrap tests could be easily obtained without using any bootstrap sampling arguments, so that resulting p-values are also based on exact probabilities of well defined extreme regions. Being systematic approaches, by taking similar approaches, researchers should be able to derive generalized tests in MANCOVA, higher-way MANOVA, and in RM MANOVA under heteroscedasticity, in which the parametric bootstrap type approaches run into difficulties.

In this article we derive two simple tests for two-way ANOVA under unequal variances requiring some constraints, which cannot be solved by classical regression formulas. We do so by taking the generalized p-value approach and provide explicit formulas to handle the constraints. The first test is an extension of fiducial one-way ANOVA test, which tends to assure the intended size of the test, but tends to be conservative, and the second test reduces that drawback, but may slightly exceed the intended size of the test. The second test is also a generalized test that is numerically equivalent to the parametric bootstrap (PB) test, which contains some unintended glitches in formulas, which we will rectify. Moreover, our approach does not require good estimators of parameters as PB approach does. Hence, one can take our approach dealing with such distributions as Weibull and Gamma that are used in analysis of lifetime data in engineering and analysis of Survival data in studies of public health. By taking similar approach, researchers are also encouraged to derive generalized tests in other ANOVA applications, such as higher-way ANOVA, ANCOVA and MANOVA under heteroscedasticity, especially in Mixed Effects Model applications such as repeated measures (RM) ANOVA, where the PB approach run into difficulties.

In the recent past, the CAT procedure introduced by Pal, Lim, and Ling has been successfully applied to obtain improved testing procedures in numerous applications. Having seen such results, practitioners may resort to the CAT procedure in all testing problems, assuming that improvement is assured in all applications. To avoid such myths, in this article, we present an important class of applications, where the CAT test performs poorly, and then discuss the type of applications where CAT procedure could be accepted or avoided. However, this does not mean that it is not possible to develop improved tests by taking the CAT approach, as we show in this article by employing the LRT statistic instead of ML/REML-based CAT tests, as authors of the original CAT article also now advocate. In fact, in terms of the Type I error, LRT-based CAT test performed well among tests we studied, except when k, the number of groups in one-way layout is small, in which case the generalized p-value-based tests can be employed. We believe this note will encourage further research to take full benefits of the CAT approach, in such problems as higher way ANOVA and mixed-effects regression models, for which generalized tests are currently available.

The purpose of this note is to derive simple testing procedures for ANOVA under heteroscedasticity by a single approach that are equivalent to the prior art in the literature obtained by the Parametric Bootstrap and the Generalized Fiducial approach. By similar approach, researchers are encouraged to derive generalized tests in other applications, as alternative to parametric bootstrap tests and fiducial tests, including ANCOVA and MANOVA under heteroscedasticity, especially in Mixed Model applications, where the bootstrap approach fails.