Jill S. Cannon

Quality rating and improvement systems (QRISs) are designed to assess, improve, and communicate the quality of early care and education programs. Using administrative data, we investigated whether providers that participated in a county-level QRIS saw increases to their global quality (tier) rating, domain-specific quality ratings, and scores on an observation measure of classroom quality. Research Findings: On average, participating providers increased their overall tier ratings; this pattern was particularly true for providers with lower initial tier ratings. We found more consistent evidence of increases on indicators of structural quality components (e.g., teacher and director qualifications), and limited evidence of improvement on process quality indicators (e.g., measure of teacher–child interaction). This finding suggests structural quality improvements may be driving global tier rating changes. In addition, providers saw average increases on a measure of classroom quality, with larger increases seen in domains that measure classroom organization and emotional support. Practice and Policy: We discuss implications for how QRISs can promote quality improvement for participating providers and suggest avenues for future research.

RAND researchers use cost data collection and cost modeling to estimate the cost of a high-quality publicly funded pre-kindergarten program. They outline the challenges in understanding full program costs and the implications for policymakers.

RAND researchers conducted a developmental evaluation of Quality Start Los Angeles (QSLA), the voluntary quality rating and improvement system for early learning providers in Los Angeles County. The goal of the evaluation was to determine whether selected components of the QSLA model were feasible, appropriate, and being implemented as designed.

Implementation support interventions have helped organizations implement programs with quality and obtain intended outcomes. For example, a recent randomized controlled trial called Preparing to Run Effective Programs (PREP) showed that an implementation support intervention called Getting To Outcomes (GTO) improved implementation of an evidence-based substance use prevention program (CHOICE) run in community-based settings. However, more information is needed on how these interventions affect organizational barriers and facilitators of implementation. This paper aims to identify differences in implementation facilitators and barriers in sites conducting a substance use prevention program with and without GTO. PREP is a cluster-randomized controlled trial testing GTO, a two-year implementation support intervention, in Boys & Girls Clubs. The trial compares 15 Boys & Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 Boys & Girls Club sites implementing CHOICE supported by GTO (intervention group). All sites received CHOICE training. Intervention sites also received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE, and then evaluate and make quality improvements to CHOICE implementation using feedback reports summarizing their data. Following the second year of CHOICE and GTO implementation, all sites participated in semi-structured interviews to identify barriers and facilitators to CHOICE implementation using the Consolidated Framework for Implementation Research (CFIR). This paper assesses the extent to which these facilitators and barriers differed between intervention and control group. Intervention sites had significantly higher average ratings than control sites for two constructs from the CFIR process domain: planning and reflecting and evaluating. At the same time, intervention sites had significantly lower ratings on the culture and available resources constructs. Findings suggest that strong planning, evaluation, and reflection—likely improved with GTO support—can facilitate implementation even in the face of perceptions of a less desirable implementation climate. These findings highlight that implementation support, such as GTO, is likely to help low-resourced community-based organizations improve program delivery through a focus on implementation processes. Trial Registration This project is registered at ClinicalTrials.gov with number NCT02135991 (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was first registered May 12, 2014.

Community organizations can have difficulty implementing evidence-based prevention programs. More research is needed on implementation support interventions designed to help these organizations implement programs with quality. Preparing to Run Effective Programs (PREP) is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 15 Boys and Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 similar sites implementing CHOICE supported by GTO (intervention group). PREP replicates a previous GTO study that had the same design, but featured a teen pregnancy prevention program instead. All sites received typical CHOICE training. Fourteen intervention sites received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE. Then, this group was trained on evaluation and quality improvement steps of GTO using feedback reports summarizing their own data, which yielded revised plans for subsequent implementation of CHOICE. This paper presents results regarding GTO’s impact on CHOICE fidelity (adherence, quality of delivery, dosage) and the proximal outcomes of the youth participants (aged 10–14)—attitudes and intentions regarding cigarettes, alcohol, and marijuana use. Fidelity was assessed at all sites by observer ratings and attendance logs. Proximal outcomes were assessed via survey at baseline, 3, and 6 months. After 1 year, fidelity and proximal outcomes were similar between Intervention and control groups. After 2 years (which included GTO quality improvement activities that took place between years 1 and 2), intervention sites had higher ratings of CHOICE adherence and quality of delivery (dosage remained similar). Proximal outcomes did not differ between groups in either year, although there was universally high endorsement of prosocial responses to those outcomes from the start. Findings suggest that systematic implementation support provided by GTO can help community organizations achieve better fidelity. Findings replicate the implementation results from a previous GTO study using the same design, but with a different evidence-based program and different fidelity measures. Although proximal outcomes did not change, in large part due to ceiling effects, the implementation findings suggest GTO can support a variety of programs. This project is registered at ClinicalTrials.gov with number NCT02135991 . The trial was first registered on May 12, 2014.

Growth in federal, state, and private funding is fueling the initiation of home visiting programs around the country. As communities expand home visiting programs, they need information to help them successfully start up new sites. This paper documents lessons learned about home visiting installation and initial implementation from the replication of the First Born^® Program in six counties in New Mexico. Specifically, we examine how well sites met staffing, family referral and enrollment, program model fidelity, and financing goals in the first year of providing services. Data come from semi-structured interviews with program staff and document review. The findings are likely to be valuable to a wide spectrum of communities starting or expanding home visiting services, as well as to public and private funders of programs.

Growth in federal, state and private funding is fueling the initiation of home visiting programs around the country. As communities expand home visiting programs, they need information about how they can successfully start up new sites. This paper proposes measures of successful home visiting program implementation and identifies factors that promote successful implementation or serve as barriers to program initiation. It focuses on lessons learned from the replication of the First Born® Program in six counties in New Mexico. Specifically, it examines how well sites met staffing, family referral and enrollment, program fidelity, and financing goals in the first year of providing services. Data come from semi-structured interviews with senior program staff and program documentation. The findings are likely to be valuable to a wide spectrum of communities starting or expanding home visiting services, as well as to public and private funders of programs.