Tor Ole. B. Odden

Generative AI is rapidly reshaping how physicists teach, learn, and conduct research, yet little is known about how physics faculty are responding to these changes. We interviewed 12 physics professors at a major Scandinavian research university to explore their uses and perceptions of Generative AI (GenAI) in both teaching and research. Using the theoretical framework of epistemic framing, we conducted a thematic analysis that identified 19 overlapping practices, ranging from coding and literature review to assessment and feedback. From these practices, we derived six overlapping epistemic frames through which professors make sense of GenAI: as a threat to genuine learning and assessment, a source of knowledge, a discussion partner, a text-processing tool, a coding tool, and a labor-saving device. While the latter five position GenAI as a useful tool in the physicists' toolbox, the threat frame represented an overarching concern that colored all other frames. These findings reveal how GenAI is beginning to transform what it means to be a physicist, highlighting both opportunities for innovation and challenges for academic integrity and learning.

Computation has become an integral part of physics research. However, little is known about how students learn to productively use computation as a tool beyond the introductory level, especially as they transition into physics research. In this study, we apply the theory of physics computational literacy and the novice-expert framework to describe the development of expertise in computational physics, as students transition from novice to journeyman computational physicists. We base this description on a thematic analysis of interviews with 13 computational physics master’s students with extensive experience using computation. We first describe the most important elements driving the development of computational physics expertise, identifying two distinct transitions of competence during their studies, driven by experience with large computational projects and professional research. We then present an overview of the various skills students attain on this path toward the journeyman level of computational physics expertise. Based on these results, we argue for the need to assist students in collaborative coding and in the learning of new tools, as well as for the importance of large, scaffolded, computational projects in helping students develop the advanced skills needed for computational research.

This study systematically explores the thematic evolution of articles in and journals during a critical period in modern history, from the Cold War era to the prepandemic world (1966–2019). To contextualize the analysis, we first present an overview of historical events and developments that have characterized physics education in the United States and Europe since the 1950s. Using a natural-language-processing-based inductive topic modeling approach, we then identify recurring themes that have shaped physics education literature, including content-based topics, teaching methodologies, laboratory practices, curriculum development, and the influence of physics education research (PER). Our findings reveal both overarching trends and distinct thematic preferences between the journals. has historically emphasized curriculum structures, social aspects of education, and interdisciplinary connections, whereas has focused more on pedagogical strategies, demonstrations, and practical teaching tools. In the last three decades, both journals have increasingly incorporated discussions on technology, computation, and PER-driven instructional practices. By tracing these shifts over five decades, this study provides a broader perspective on how physics education has responded to changing educational priorities, technological advancements, and research developments.

Generative AI is rapidly reshaping how physicists teach, learn, and conduct research, yet little is known about how physics faculty are responding to these changes. We interviewed 12 physics professors at a major Scandinavian research university to explore their uses and perceptions of Generative AI (GenAI) in both teaching and research. Using the theoretical framework of epistemic framing, we conducted a thematic analysis that identified 19 overlapping practices, ranging from coding and literature review to assessment and feedback. From these practices, we derived six overlapping epistemic frames through which professors make sense of GenAI: as a threat to genuine learning and assessment, a source of knowledge, a discussion partner, a text-processing tool, a coding tool, and a labor-saving device. While the latter five position GenAI as a useful tool in the physicists' toolbox, the threat frame represented an overarching concern that colored all other frames. These findings reveal how GenAI is beginning to transform what it means to be a physicist, highlighting both opportunities for innovation and challenges for academic integrity and learning.

Understanding how Physics Education Research (PER) has changed and matured can help researchers and materially improve physics teaching and learning. The advent of large language models (LLMs) and embedding-based NLP techniques opens new avenues for this work by enabling the analysis of large corpora with minimal manual coding. This preliminary study builds off the work of [T. O. B. Odden et al., Physical Review Physics Education Research 20, 020151 (2024)]. We explore the use of text embeddings and retrieval-augmented generation (RAG)-like methods without generative AI to analyze themes across a random sample of 94 articles from major PER journals. Specifically, we examine how two methodological choices affect topic modeling performance: (1) representing articles using a single embedding versus multiple sentence-level embeddings, and (2) deriving topic centroids from representative texts versus researcher-defined queries. All results are evaluated against a human-coded dataset focused on four overarching thematic categories: teacher-centered, student-centered, physics content, and journal business. Our findings inform best practices for researchers seeking scalable, interpretable, and low-barrier approaches to literature analysis in PER.

Qualitative analysis of open-ended survey responses is a commonly-used research method in the social sciences, but traditional coding approaches are often time-consuming and prone to inconsistency. Existing solutions from Natural Language Processing such as supervised classifiers, topic modeling techniques, and generative large language models have limited applicability in qualitative analysis, since they demand extensive labeled data, disrupt established qualitative workflows, and/or yield variable results. In this paper, we introduce a text embedding-based classification framework that requires only a handful of examples per category and fits well with standard qualitative workflows. When benchmarked against human analysis of a conceptual physics survey consisting of 2899 open-ended responses, our framework achieves a Cohen's Kappa ranging from 0.74 to 0.83 as compared to expert human coders in an exhaustive coding scheme. We further show how performance of this framework improves with fine-tuning of the text embedding model, and how the method can be used to audit previously-analyzed datasets. These findings demonstrate that text embedding-assisted coding can flexibly scale to thousands of responses without sacrificing interpretability, opening avenues for deductive qualitative analysis at scale.

We propose a technique for performing deductive qualitative data analysis at scale on text-based data. Using a natural language processing technique known as text embeddings, we create vector-based representations of texts in a high-dimensional meaning space within which it is possible to quantify differences in meaning as vector distances. To apply the technique, we build off prior work that used topic modeling via latent Dirichlet allocation to thematically analyze 18 years of the Physics Education Research Conference Proceedings literature. We first extend this analysis through 2023. Next, we create embeddings of all texts and, using representative articles from the 10 topics found by the LDA analysis, define centroids in the meaning space. We calculate the distances between every article and centroid and use the inverted, scaled distances between these centroids and articles to create an alternate topic model. We benchmark this model against the LDA model results and show that this embeddings model recovers most of the trends from that analysis. Finally, to illustrate the versatility of the method, we define eight new topic centroids derived from a review of the physics education research literature by Docktor and Mestre and reanalyze the literature using these researcher-defined topics. Based on these analyses, we critically discuss the features, uses, and limitations of this method and argue that it holds promise for flexible deductive qualitative analysis of a wide variety of text-based data that avoids many of the drawbacks inherent to prior NLP methods. Published by the American Physical Society 2024