Yekaterina Kharitonova

The rise of generative artificial intelligence (GenAI) has sparked a rapid change in computing curricula and teaching approaches. GenAI coding tools can accurately complete assignments, answer test questions, and perform other tasks traditionally associated with learning programming, especially at the introductory level. Because GenAI is still so new, researchers investigating student usage of GenAI have used informal rubrics and questionnaires. To advance, the field needs validated instruments that measure student perception and use of GenAI. This paper presents the development and initial validation of an instrument to measure self-efficacy while using GenAI to learn programming. Self-efficacy is an important construct in education research because it robustly correlates with student success, across disciplines and ages, including undergraduate computing education. Computing education researchers have presented several validated self-efficacy instruments, most recently by Steinhorst et al. in 2020. Critically, this instrument was created before the rise of GenAI’s popularity in 2022. To complement this instrument, we created a GenAI scale similar in style to the Steinhorst self-efficacy instrument, consisting originally of 11 items and revised to 5 items. We report two important findings in this paper. First, we found strong support for the validity of the existing Steinhorst instrument in a new context, specifically an introductory programming course that fully integrates GenAI. Second, the new GenAI scale shows strong internal reliability, discriminant validity with items in the Steinhorst subscales, and criterion validity with students’ GenAI usage patterns. Based on statistical analysis and cognitive probing interviews, we argue for the validity of the five-item scale to measure students’ GenAI self-efficacy in the context of programming.

Motivation : In computing, Impostor Phenomenon (IP) has been viewed as a problem for many years, but little research has been done to show its prevalence. In 2020, IP in computing began to be explored at single institutions [ 68 ]. The results showed that IP is prevalent among undergraduate and graduate students in computing courses and that the rates of IP are higher for women. In 2022, these results were reaffirmed with a replication study including two institutions [ 82 ]. This is concerning due to the negative effects correlated with people who experience IP such as low self-esteem [ 19 , 37 ] and anxiety [ 21 , 38 ]. Objectives : This study aims to replicate these previous findings at a considerably larger scale to determine whether similar results are observed across institutions. To support future work, we conduct an exploratory analysis of student demographics, course factors, and institutional factors to gain insight into factors that may be associated with higher levels of IP among students. Methods : A survey consisting of Clance’s IP scale (CIPS) and questions on students’ demographic and background information was given at 18 institutions. Higher CIPS scores indicate more IP experiences. Differences in CIPS scores were analyzed based on students’ demographics and background information (gender, race/ethnicity, transfer status, and chosen degree program), course factors (introductory computing courses vs. non-introductory computing courses, upper- vs. lower-division), and institutional factors (size of the institution, public vs. private, teaching- vs. research-centric). Results : Our results are consistent with previous findings that IP is prevalent among students in computing courses and that women have significantly higher CIPS scores of IP than men in computing, and that traditionally marginalized race/ethnicity status in computing and chosen degree program do not have an observable impact. In terms of course factors, we do not see a difference in scores based on whether students are enrolled in a lower- or upper-division course. We see that students enrolled in introductory computing (CS1) courses have statistically significant higher scores than students outside of CS1 courses. In terms of institutional factors, students in computing courses at public institutions have statistically significantly higher scores than students at private institutions. Students at medium-sized institutions have statistically significantly higher scores than students at small or large institutions. We do not find any difference based on whether an institution is teaching- or research-centric. Discussion : These results suggest that IP is prevalent in computing courses across the entire curriculum and across different types of institutions. Differences in demographic groups are consistent with prior work in computing, specifically higher rates among women, suggesting IP may be worth further inquiry as a potential factor in the gender participation gap in computing.