Bailey Kimmel

Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Previous research has shown that novices can encounter multiple metacognitive difficulties while programming, such as forming incorrect conceptual models of the problem or having a false sense of progress after testing their solution. Novices are typically unaware of how these difficulties are hindering their progress. Meanwhile, many novices are now programming with generative AI (GenAI), which can provide complete solutions to most introductory programming problems, code suggestions, hints for next steps when stuck, and explain cryptic error messages. Its impact on novice metacognition has only started to be explored. Here we replicate a previous study that examined novice programming problem solving behavior and extend it by incorporating GenAI tools. Through 21 lab sessions consisting of participant observation, interview, and eye tracking, we explore how novices are coding with GenAI tools. Although 20 of 21 students completed the assigned programming problem, our findings show an unfortunate divide in the use of GenAI tools between students who did and did not struggle. Some students who did not struggle were able to use GenAI to accelerate, creating code they already intended to make, and were able to ignore unhelpful or incorrect inline code suggestions. But for students who struggled, our findings indicate that previously known metacognitive difficulties persist, and that GenAI unfortunately can compound them and even introduce new metacognitive difficulties. Furthermore, struggling students often expressed cognitive dissonance about their problem solving ability, thought they performed better than they did, and finished with an illusion of competence. Based on our observations from both groups, we propose ways to scaffold the novice GenAI experience and make suggestions for future work.

Generative AI is changing the way that many disciplines are taught, including computer science. Researchers have shown that generative AI tools are capable of solving programming problems, writing extensive blocks of code, and explaining complex code in simple terms. Particular promise has been shown in using generative AI to enhance programming error messages. Both students and instructors have complained for decades that these messages are often cryptic and difficult to understand. Yet recent work has shown that students make fewer repeated errors when enhanced via GPT-4. We extend this work by implementing feedback from ChatGPT for all programs submitted to our automated assessment tool, Athene, providing help for compiler, run-time, and logic errors. Our results indicate that adding generative AI to an automated assessment tool does not necessarily make it better and that design of the interface matters greatly to the usability of the feedback that GPT-4 provided.

Ultraviolet light can affect the appearance and medical condition of the human skin by triggering biophysical processes such as erythema, melanogenesis, photoaging and carcinogenesis. The evolution of these processes is related to the amount of ultraviolet light absorbed by skin pigments. This amount may vary with the wavelength and path length of the radiation that is propagated within the skin tissues. For many years, biomedical researchers have been investigating the propagation of ultraviolet light in skin tissues through Monte Carlo simulations. The scattering of the incident radiation by tissue internal structures, a key component in this process, is usually approximated by functions without a plausible connection with the underlying physical phenomena. In this paper, we examine the origins of such an approach, and question its generalized use with respect to wavelengths and biological materials for which there is no supporting data available. Furthermore, we perform comparisons to demonstrate that the accuracy and predictability of Monte Carlo simulations of ultraviolet propagation in skin tissues can be improved by using a data-driven approach to represent the scattering profile of these tissues.