T. Chris Riley-Tillman

Research has supported the applied use of Direct Behavior Rating Single-Item Scale (DBR-SIS) targets of "academic engagement" and "disruptive behavior" for a range of purposes, including universal screening and progress monitoring. Though useful in evaluating social behavior and externalizing problems, these targets have limited utility in evaluating emotional behavior and internalizing problems. Thus, the primary purpose of this study was to support the initial development and validation of a novel DBR-SIS target of "unhappy," which was intended to tap into the specific construct of depression. A particular focus of this study was on the novel target's utility within universal screening. A secondary purpose was to further validate the aforementioned existing DBR-SIS targets. Within this study, 87 teachers rated 1,227 students across two measures (i.e., DBR-SIS and the Teacher Observation of Classroom Adaptation-Checklist [TOCA-C]) and time points (i.e., fall and spring). Correlational analyses supported the test-retest reliability of each DBR-SIS target, as well as its convergent and discriminant validity across concurrent and predictive comparisons. Receiver operating characteristic (ROC) curve analyses further supported (a) the overall diagnostic accuracy of each target (as indicated by the area under the curve [AUC] statistic), as well as (b) the selection of cut scores found to accurately differentiate at-risk and not at-risk students (as indicated by conditional probability statistics). A broader review of findings suggested that across the majority of analyses, the existing DBR-SIS targets outperformed the novel "unhappy" target. Impact and Implications Research suggests that although many students exhibit within internalizing concerns, schools struggle to identify them in a timely manner. The results of the study indicate how the Direct Behavior Rating Single-Item Scales (DBR-SIS) can be used to identify students exhibiting such concerns.

Responsive service delivery frameworks rely on the use of screening approaches to identify students in need of support and to guide subsequent assessment and intervention efforts. However, limited empirical investigations have been directed to informing how often screening should occur for social, emotional, and behavioral difficulties in school settings. The purpose of the current study was to evaluate the stability of risk status on 3 different screening instruments across 3 administrations across the course of a school year. A total of 1,594 students had complete screening data across 3 time points, corresponding to a total of 187 teachers from 22 different public schools located within the northeastern and midwestern United States. Across measures, we examined patterns of risk across time points and investigated the utility of (a) different screening schedules (fall, winter, and spring) and (b) borderline screening when conducting multiple screenings per year. Results indicated that a large proportion of students exhibited stable risk patterns across time points and suggested that borderline screening may be a viable alternative for schools with limited resources. Implications for practice and future research are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

This study examined the treatment sensitivity of Direct Behavior Rating–Single Item Scales (DBR-SIS) in response to an evidence-based intervention delivered in a single-case, multiple-baseline design. DBR-SIS was used as a formative assessment in conjunction with a frequently used intervention in schools, a Daily Behavior Report Card (DRC). The intervention and concurrent assessment were conducted by five teachers in a rural Midwestern elementary school with five male students displaying mild to moderate behavioral challenges in the classroom. Study findings indicated DBR-SIS displays appropriate treatment sensitivity following intervention implementation. Agreement in the documentation of response and nonresponse to intervention implementation between DBR-SIS and systematic direct observation (SDO) data was evident across visual and empirical analyses. In addition, through a multiple-baseline design, this study documented negligible to no change in student behavior following implementation of a DRC in an applied classroom setting. These findings support previous calls for continued examination of the forms and components of DRC employed in schools. Finally, the study found educators rated the use of a combined DRC intervention and progress monitoring with DBR-SIS as favorable.

Counterbalancing treatment order in experimental research design is well established as an option to reduce threats to internal validity, but in educational and psychological research, the effect of varying the order of multiple tests to a single rater has not been examined and is rarely adhered to in practice. The current study examines the effect of test order on measures of student behavior by teachers as raters utilizing data from a behavior measure validation study. Using multilevel modeling to control for students nested within teachers, the effect of rating an earlier measure on the intercept or slope of a later behavior assessment was statistically significant in 22% of predictor main effects for the spring test period. Test order effects had potential for high stakes consequences with differences large enough to change risk classification. Results suggest that researchers and practitioners in classroom settings using multiple measures evaluate the potential impact of test order. Where possible, they should counterbalance when the risk of an order effect exists and report justification for the decision to not counterbalance. (PsycINFO Database Record

Recent studies have suggested single case design (SCD) intervention research is subject to publication bias, wherein studies are more likely to be published if they possess large or statistically significant effects and use rigorous experimental methods. The nature of SCD and the purposes for which it might be used could suggest that large effects and rigorous methods should not always be expected. The purpose of the current paper is to propose and describe a theory-driven cycle of SCD intervention research. The proposed SCD-specific cycle serves several purposes including (a) defining the purposes for which SCD research might be adopted, (b) specifying the types of evidence to be collected in establishing an intervention for applied use, and (c) illustrating the phases of SCD-based intervention research (i.e., development, efficacy, effectiveness, contextualization, and implementation). The proposed model is intended to serve as an intermediary between theory and research, facilitating the consideration of theory-driven expectations when selecting particular SCDs (e.g., reversal, multiple baseline) and interpreting SCD data. Next steps and future directions for research are included.

Over the past 3 decades, there has been an unprecedented increase in students identified as eligible for special education as a result of students meeting criteria for autism spectrum disorder (ASD). The increasing number of students with ASD in the schools presents significant challenges to teachers, school psychologists, and other school professionals working with this population. Although there is considerable research addressing assessment, identification, and support services for children with ASD, there is a need for further research focused on these topics within the school context. Employing a diverse array of methodologies, the articles in this special topic section address several gaps in the literature, including (a) the application of evidence-based programs within the school context, (b) the social validity of well-established evidence-based practices with both parents and educators, (c) the assessment of social communication, (d) intervention and assessment of spatial and body awareness for children with ASD, (e) the use of peer-mediated discreet trial training, and (f) discrepancies across informants for both externalizing and internalizing symptoms associated with ASD. The results of these studies provide school psychologists and other education professionals with specific directions for advocacy and service delivery that aim to enhance school outcomes for students with ASD. (PsycINFO Database Record

Multi-tiered problem-solving models that focus on promoting positive outcomes for student behavior continue to be emphasized within educational research. Although substantial work has been conducted to support systems-level implementation and intervention for behavior, concomitant advances in behavior assessment have been limited. This is despite the central role that data derived from behavior assessment methods must play in making defensible multi-tiered decisions such as those for screening and progress monitoring. In this commentary, the role of assessment in the evidence-based practice movement is described, alongside necessary features of behavior assessment methods utilized in multi-tiered systems. The relevance of these features to articles in this special issue is described. Finally, observations and suggestions for future directions regarding the current state of behavior assessment in educational research are offered.

The purpose of this investigation was to evaluate the reliability of Direct Behavior Ratings-Social Competence (DBR-SC) ratings. Participants included 60 students identified as possessing deficits in social competence, as well as their 23 classroom teachers. Teachers used DBR-SC to complete ratings of 5 student behaviors within the general education setting on a daily basis across approximately 5 months. During this time, each student was assigned to 1 of 2 intervention conditions, including the Social Competence Intervention-Adolescent (SCI-A) and a business-as-usual (BAU) intervention. Ratings were collected across 3 intervention phases, including pre-, mid-, and postintervention. Results suggested DBR-SC ratings were highly consistent across time within each student, with reliability coefficients predominantly falling in the .80 and .90 ranges. Findings further indicated such levels of reliability could be achieved with only a small number of ratings, with estimates varying between 2 and 10 data points. Group comparison analyses further suggested the reliability of DBR-SC ratings increased over time, such that student behavior became more consistent throughout the intervention period. Furthermore, analyses revealed that for 2 of the 5 DBR-SC behavior targets, the increase in reliability over time was moderated by intervention grouping, with students receiving SCI-A demonstrating greater increases in reliability relative to those in the BAU group. Limitations of the investigation as well as directions for future research are discussed herein.

The purpose of this study was to investigate the impact of two different Direct Behavior Rating–Single Item Scale (DBR-SIS) formats on rating accuracy. A total of 119 undergraduate students participated in one of two study conditions, each utilizing a different DBR-SIS scale format: one that included percentage of time anchors on the DBR-SIS scale and an explicit reference to duration of the target behavior (percent group) and one that did not include percentage anchors nor a reference to duration of the target behavior (no percent group). Participants viewed nine brief video clips and rated student behavior using one of the two DBR-SIS formats. Rating accuracy was determined by calculating the absolute difference between participant ratings and two criterion measures: systematic direct observation scores and DBR-SIS expert ratings. Statistically significant differences between groups were found on only two occasions, pertaining to ratings of academically engaged behavior. Limitations and directions for future research are discussed.