Jason Ravitz

Until recently, computer science (CS) has been predominantly taught at upper-secondary or tertiary levels. Lately, however, CS curricula have been introduced into school systems from the very first year of school. In this paper, we undertake a participatory research approach, using focus group discussions between a group of experts in the field of evaluation and assessment at the primary level (K-5). The group considered the evaluation and assessment measures they have used, what their current needs are and how the CS education community can move towards meeting those needs. We present the discussion results as a position paper, situated in the context of broader education research. The experts identified three key priorities for the education research community: creating a universal taxonomy of assessment in the primary grades (K-5), creating measurements of student progression and growth over time, and creating culturally relevant evaluations and assessments. Through identifying key priorities, this work provides direction for urgently needed resource development and research directions for K-5 evaluation and assessment.

Google's educator professional development (PD) grants (formerly Computer Science for High School [CS4HS]) provide annual funding to education nonprofits that design and deliver Computer Science (CS) PD to educators in their local communities. As CS education is an emerging field, many education stakeholders can be ill-equipped to identify CS PD needs, evaluate options, and assess educator and student outcomes. As a result educators may participate in CS PD that fails to address their needs, which worsens equity gaps in CS education. Therefore, models of evaluating CS PD programs and outcomes are critical to equitable CS education. This paper provides an update on earlier research findings from 2014 with data from the 2015 and 2016 evaluation cycles, as well as updates to our evaluation measures and methodology.

This article compares self-reported learning gains and experiences of teachers in four professional development courses funded through Google’s 2014 Computer Science for High School program. The courses were designed and taught independently at four universities and started late enough in the year to participate in our pre-post study. Two of the courses used a face-to-face approach, one was online only, and one used a hybrid format. Analyses from 314 pre-surveys and 129 post-surveys indicate CS teachers are far from homogenous, suggesting that some customization may benefit professional development. We also saw a stronger sense of community in the two face-to-face courses. Among the outcomes we measured, teacher concerns (Hall and Hord 1977) were more sensitive to change than our measures of self-efficacy, outcome expectations, readiness, or beliefs. Findings illustrate the variety of CS teacher professional development experiences and the need to study the best ways to scale effective CS teacher education.

The importance of computer science educational opportunities for girls cannot be understated, and strong evaluative practices are critical for ensuring that these opportunities are of high quality and meet girls' needs. In this poster presentation, participants will learn how a nonprofit and a corporate partner designed a program to build the evaluative capacity of participating nonprofits--the CS OPEN initiative. Presenters will discuss the guiding principles that shaped this initiative and share the challenges to and successes of developing an evaluation capacity building network that supports a diverse set of 12 grantees. Session attendees will learn from the designers and evaluation experts within this network: 1) how this evaluation capacity building partnership was developed and grantees were selected, 2) grantees' experiences and challenges in designing and implementing evaluations, 3) the resources and efforts the partners have used to develop grantees' evaluative capacity and to build the network, and 4) the impacts of these efforts on grantees evaluative capacity. Resources utilized by the CS OPEN grantees will be displayed with ratings and highlights of their growth in evaluative capacity. The poster will highlight major successes and lessons learned for implementing an evaluative capacity building grant initiative.

With the growth of informal and out of school educational focus on increasing female participation in CS, many of us are asking critical questions about program development, participant learning, and program impact. Which programs are having success reaching and encouraging girls to pursue computer science? What stumbling blocks are encountered? How successfully are programs engaging girls with different STEM orientations or different racial/ethnic/economic backgrounds? How successful are existing CS programs in pivoting to be more accessible to females? Many nonprofit organizations cannot effectively answer these questions because they lack the capacity for program evaluation. Some organizations delay putting evaluation into place, or use a one-dimensional evaluation approach (i.e., a single survey) even though ongoing and comprehensive evaluation results are crucial to program success. Building internal evaluation capacity can help organizations meet their most critical evaluation needs and build systems of continuous improvement. This BOF session will engage participants in discussing their challenges in designing and implementing program evaluations. Participants will have the opportunity to share strategies, evaluative resources, and challenges in building evaluative capacity and conducting effective evaluations. Discussion leaders will share resources developed as part of an evaluation capacity building initiative with informal CS education programs--CS Outreach Program Evaluation Network (CS OPEN--a collaboration between the National Girls Collaborative and Google). As an outcome of this BOF discussion, we hope to collectively identify some lessons learned and next steps for building evaluative capacity around informal CS education.

Recent studies on the impact of CS teacher professional development (PD) opportunities have highlighted areas of improvement to effectively grow the number of well-prepared CS teachers. In this BoF Google will share data from CS4HS, an annual program that awards funding to research institutions and education nonprofits globally for the purpose of developing innovative and scalable CS PD that maps to local standards and demands. These data will serve as a starting point to engage the attendees in a wide-ranging and action-oriented discussion on ways the CS education community can improve our PD processes and results.

Increasing women's participation in computer science is a critical workforce and equity concern. The technology industry has committed to reversing negative trends for women in computer science as well as engineering and information technology \"computing\" fields. Building on previously published research, this paper identifies factors that influence young women's decisions to pursue computer science-related degrees and the ways in which these factors differ for young men. It is based on a survey of 1,739 high school students and recent college graduates. Results identified encouragement and exposure as the leading factors influencing this critical choice for women, while the influence of these factors is different for men. In particular, the influence of family is found to play a critical role in encouragement and exposure, and outreach efforts should focus on ways to engage parents.