Sample size requirements for the design of reliability studies: precision consideration
Author:
- 12 December 2013
- journal article
- Published by Springer Nature in Behavior Research Methods
- Vol. 46 (3) , 808-822
Abstract
In multilevel modeling, the intraclass correlation coefficient based on the one-way random-effects model is routinely employed to measure the reliability or degree of resemblance among group members. To facilitate the advocated practice of reporting confidence intervals in future reliability studies, this article presents exact sample size procedures for precise interval estimation of the intraclass correlation coefficient under various allocation and cost structures. Although the suggested approaches do not admit explicit sample size formulas and require special algorithms for carrying out iterative computations, they are more accurate than the closed-form formulas constructed from large-sample approximations with respect to the expected width and assurance probability criteria. This investigation notes the deficiency of existing methods and expands the sample size methodology for the design of reliability studies that have not previously been discussed in the literature.Keywords
This publication has 25 references indexed in Scilit:
- Guide for calculating and interpreting effect sizes and confidence intervals in intellectual and developmental disability research studiesJournal of Intellectual & Developmental Disability, 2012
- Effect size estimates: Current use, calculations, and interpretation.Journal of Experimental Psychology: General, 2012
- An effect size primer: A guide for clinicians and researchers.Professional Psychology: Research and Practice, 2009
- Sample size requirements for estimating intraclass correlations with desired precisionStatistics in Medicine, 2002
- Recruitment strategies in a cluster randomized trial – cost implicationsStatistics in Medicine, 2002
- Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficientStatistics in Medicine, 2001
- Sample size requirements for interval estimation of the intraclass kappa statisticCommunications in Statistics - Simulation and Computation, 1999
- A cost‐function approach to the design of reliability studiesStatistics in Medicine, 1987
- A note on the accuracy of fisher's approximation to the large sample variance of an intraclass correlationaCommunications in Statistics - Simulation and Computation, 1983
- On various intraclass correlation reliability coefficients.Psychological Bulletin, 1976