How Many Tiers Do We Need? Type I Errors and Power in Multiple Baseline Designs
Authors:
Open Access
- 29 July 2020
- journal article
- research article
- Published by Springer Nature in Perspectives on Behavior Science
- Vol. 43 (3) , 605-616
Abstract
Design quality guidelines typically recommend that multiple baseline designs include at least three demonstrations of effects. Despite its widespread adoption, this recommendation does not appear grounded in empirical evidence. The main purpose of our study was to address this issue by assessing Type I error rate and power in multiple baseline designs. First, we generated 10,000 multiple baseline graphs, applied the dual-criteria method to each tier, and computed Type I error rate and power for different number of tiers showing a clear change. Second, two raters categorized the tiers for 300 multiple baseline graphs to replicate our analyses using visual inspection. When multiple baseline designs had at least three tiers and two or more of these tiers showed a clear change, the Type I error rate remained adequate (< .05) while power also reached acceptable levels (> .80). In contrast, requiring all tiers to show a clear change resulted in overly stringent conclusions (i.e., unacceptably low power). Therefore, our results suggest that researchers and practitioners should carefully consider limitations in power when requiring all tiers of a multiple baseline design to show a clear change in their analyses.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Detecting False Positives in A-B DesignsBehavior Modification, 2012
- Single-Case Intervention Research Design StandardsRemedial and Special Education, 2012
- Single-case experimental designs: A systematic review of published research and current standards.Psychological Methods, 2012
- Characteristics of single-case designs used to assess intervention effects in 2008Behavior Research Methods, 2011
- CONSISTENT VISUAL ANALYSES OF INTRASUBJECT DATAJournal of Applied Behavior Analysis, 2010
- A meta-analysis of single subject design writing intervention research.Journal of Educational Psychology, 2008
- AN EVALUATION OF THE CONSERVATIVE DUAL‐CRITERION METHOD FOR TEACHING UNIVERSITY STUDENTS TO VISUALLY INSPECT AB‐DESIGN GRAPHSJournal of Applied Behavior Analysis, 2007
- VISUAL AIDS AND STRUCTURED CRITERIA FOR IMPROVING VISUAL INSPECTION AND INTERPRETATION OF SINGLE‐CASE DESIGNSJournal of Applied Behavior Analysis, 2003
- INCONSISTENT VISUAL ANALYSES OF INTRASUBJECT DATAJournal of Applied Behavior Analysis, 1979
- SOME CURRENT DIMENSIONS OF APPLIED BEHAVIOR ANALYSIS1Journal of Applied Behavior Analysis, 1968