Gerd Kortemeyer
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Gerd Kortemeyer has a total of 74 co-authors. Here are the top co-authors by number of shared publications:
- Wolfgang Bauer
Michigan State University
5 shared publications Christine Kortemeyer
Lucerne University of Applied Sciences and Arts, 6002 Lucerne, Switzerland
2 shared publications- Claudia Merki
Unit for Teaching and Learning, ETH Zurich, Zurich, Switzerland
2 shared publications Emre Gönülateş
Michigan State Univ, Dept Counseling Educ Psychol & Special Educ, E Lansing, MI 48824 USA
2 shared publications- Julian Nöhl
ETH Zurich
2 shared publications - Nora Dittmann-Domenichini
Unit for Teaching and Learning, ETH Zurich, Zurich, Switzerland
2 shared publications
Latest Publications
- Open Access
- 21 October 2026
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 22 (2) , 020147
Testing the validity of embedding-based similarity and clustering for handwritten physics solutions
Text embeddings are increasingly used in physics education research to organize, compare, and cluster large collections of written text. Their appeal is clear: once student responses have been mapped into a vector space, similarity comparisons and clustering become computationally inexpensive. However, in assessment contexts, the relevant question is not merely whether clusters can be produced but whether the geometry of the embedding space preserves grading-relevant distinctions. We tested this premise using 992 handwritten student-problem solutions from a high-stakes engineering thermodynamics exam, transcribed into five textual representations and embedded using nine embedding mechanisms. We compared embedding similarity and embedding-based hierarchical clusters against human-assigned scores. Across models, representations, and clustering choices, embedding similarity showed a consistent but modest relationship to score similarity, and the resulting clusters were score-enriched but not score-equivalent. Experiments with a synthetic dataset suggest that this may be due to embeddings behaving like novices when categorizing physics-problem solutions; that is, their similarity geometry is strongly influenced by surface features rather than conceptual, semantic structure. These findings suggest that rubric-blind transcription and out-of-the-box, state-of-the-art embeddings can support exploratory organization and human-in-the-loop review of physics solutions, but they do not provide an unsupervised basis for grading without external validation against the assessment construct of interest.
- 21 October 2026
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 22 (2) , 020147
Open AccessTesting the validity of embedding-based similarity and clustering for handwritten physics solutions
Text embeddings are increasingly used in physics education research to organize, compare, and cluster large collections of written text. Their appeal is clear: once student responses have been mapped into a vector space, similarity comparisons and clustering become computationally inexpensive. However, in assessment contexts, the relevant question is not merely whether clusters can be produced but whether the geometry of the embedding space preserves grading-relevant distinctions. We tested this premise using 992 handwritten student-problem solutions from a high-stakes engineering thermodynamics exam, transcribed into five textual representations and embedded using nine embedding mechanisms. We compared embedding similarity and embedding-based hierarchical clusters against human-assigned scores. Across models, representations, and clustering choices, embedding similarity showed a consistent but modest relationship to score similarity, and the resulting clusters were score-enriched but not score-equivalent. Experiments with a synthetic dataset suggest that this may be due to embeddings behaving like novices when categorizing physics-problem solutions; that is, their similarity geometry is strongly influenced by surface features rather than conceptual, semantic structure. These findings suggest that rubric-blind transcription and out-of-the-box, state-of-the-art embeddings can support exploratory organization and human-in-the-loop review of physics solutions, but they do not provide an unsupervised basis for grading without external validation against the assessment construct of interest.
- Open Access
- 28 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 520-524
?Ed API: Towards a Shared API for Education Microservices
- 28 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 520-524
Open Access?Ed API: Towards a Shared API for Education Microservices
- Open Access
- 9 April 2026
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 22 (1)
When AI evaluates its own work: Validating learner-initiated, AI-generated physics practice problems
Large language models (LLMs) can now generate physics practice problems in real time, yet the educational value of these items hinges on rapid, reliable postgeneration vetting. In this exploratory study, we investigated which automated checks are both technically feasible and pedagogically meaningful when exercises are produced on demand within a chatbot interface. A cohort of 34 introductory-physics students generated and attempted 543 practice problems during exam preparation. Each item was labeled by an expert on a wide range of quality attributes and presented to the learners in pairs to record their preference. We then (i) benchmarked three commodity LLMs as “judges” against the expert labels, (ii) quantified which attributes predict student choice via random-forest models, and (iii) triangulated these results with free-form exit surveys. Only a small subset of the original metric items proved necessary to reliably address student preferences either directly or by proxy. The study demonstrates that scalable formative assessment does not require exhaustive scoring: a carefully curated core of structural and learner-visible checks is sufficient to ensure both technical soundness and user appeal. The findings provide a practical blueprint for deploying real-time, AI-generated practice in physics and other quantitative disciplines.
- 9 April 2026
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 22 (1)
Open AccessWhen AI evaluates its own work: Validating learner-initiated, AI-generated physics practice problems
Large language models (LLMs) can now generate physics practice problems in real time, yet the educational value of these items hinges on rapid, reliable postgeneration vetting. In this exploratory study, we investigated which automated checks are both technically feasible and pedagogically meaningful when exercises are produced on demand within a chatbot interface. A cohort of 34 introductory-physics students generated and attempted 543 practice problems during exam preparation. Each item was labeled by an expert on a wide range of quality attributes and presented to the learners in pairs to record their preference. We then (i) benchmarked three commodity LLMs as “judges” against the expert labels, (ii) quantified which attributes predict student choice via random-forest models, and (iii) triangulated these results with free-form exit surveys. Only a small subset of the original metric items proved necessary to reliably address student preferences either directly or by proxy. The study demonstrates that scalable formative assessment does not require exhaustive scoring: a carefully curated core of structural and learner-visible checks is sufficient to ensure both technical soundness and user appeal. The findings provide a practical blueprint for deploying real-time, AI-generated practice in physics and other quantitative disciplines.
- Open Access
- 1 January 2026
- Published by American Association of Physics Teachers (AAPT) in The Physics Teacher
- Vol. 64 (1) , 8-12
The Boiling-Frog Problem of Physics Education
- 1 January 2026
- Published by American Association of Physics Teachers (AAPT) in The Physics Teacher
- Vol. 64 (1) , 8-12
Open AccessThe Boiling-Frog Problem of Physics Education
- Open Access
- 29 October 2025
- Published by Swiss Chemical Society in Chimia
- Vol. 79 (10) , 684-688
Have to Shake It Up: STEM Education Feeling the Heat from Artificial Intelligence
General-purpose AI already correctly solves most traditional assessment problems in first-year STEM education, and it continues to become more proficient with every release. This quiet superheating of familiar practices by increasing AI capabilities will yield a messy eruption unless instructors introduce deliberate ‘nucleation sites’ and shake up the curriculum. We argue that results of science education research are now more relevant than ever since they provide insights and strategies on how to optimize human learning. Finally, we emphasize the importance of community and wellbeing – through peer instruction, studios, and brief, structured oral checks with whiteboarding – to counter loneliness and fatalism. As most of these instructional methods involve open-ended tasks, which tend to be more resource-intensive in grading and feedback than traditional, solution-oriented closed-form answer practices, AI can play an important role in assisting and supporting human educators. Thus, we illustrate how ETH Zurich’s Ethel system operationalizes these approaches through a course-grounded chatbot, formative feedback on handwritten work, on-demand practice, and grading assistance, while keeping learning human.
- 29 October 2025
- Published by Swiss Chemical Society in Chimia
- Vol. 79 (10) , 684-688
Open AccessHave to Shake It Up: STEM Education Feeling the Heat from Artificial Intelligence
General-purpose AI already correctly solves most traditional assessment problems in first-year STEM education, and it continues to become more proficient with every release. This quiet superheating of familiar practices by increasing AI capabilities will yield a messy eruption unless instructors introduce deliberate ‘nucleation sites’ and shake up the curriculum. We argue that results of science education research are now more relevant than ever since they provide insights and strategies on how to optimize human learning. Finally, we emphasize the importance of community and wellbeing – through peer instruction, studios, and brief, structured oral checks with whiteboarding – to counter loneliness and fatalism. As most of these instructional methods involve open-ended tasks, which tend to be more resource-intensive in grading and feedback than traditional, solution-oriented closed-form answer practices, AI can play an important role in assisting and supporting human educators. Thus, we illustrate how ETH Zurich’s Ethel system operationalizes these approaches through a course-grounded chatbot, formative feedback on handwritten work, on-demand practice, and grading assistance, while keeping learning human.
- Open Access
The Boiling-Frog Problem of Physics Education
It is astonishing how rapidly general-purpose AI has crossed familiar thresholds in introductory physics. Comparing outputs from successive models, GPT-5 Thinking moves far beyond the plug-and-chug tendencies seen earlier: on a classic elevator problem it works symbolically, notes when variables cancel, and verifies results; attempts to prompt novice-like behavior mainly affect tone, not method. On representation translation, the model scores 24/26 (92.3%) on TUG-Kv4.0. In a card-sorting proxy using two of my comprehensive finals (60 items), its categories reflect solution method rather than surface features. Solving those same exams, it attains 27/30 and 25/30, with most misses in ruler-based ray tracing and circuit interpretation. On epistemology, five independent CLASS runs yield 100\% favorable, indicating a simulated expert-like stance. Framed as a "boiling frog" problem, the paper argues for a decisive jump: retire credit-bearing unsupervised closed-response online assessments; grade process evidence; use paper, whiteboarding; shift weight to modeling, data, and authentic labs; require transparent, citable AI use; rebuild problem types; and lean on research-based instruction and peer learning. The opportunity is to foreground what AI cannot substitute for: modeling the world, arguing from evidence, and making principled approximations.
Open AccessThe Boiling-Frog Problem of Physics Education
It is astonishing how rapidly general-purpose AI has crossed familiar thresholds in introductory physics. Comparing outputs from successive models, GPT-5 Thinking moves far beyond the plug-and-chug tendencies seen earlier: on a classic elevator problem it works symbolically, notes when variables cancel, and verifies results; attempts to prompt novice-like behavior mainly affect tone, not method. On representation translation, the model scores 24/26 (92.3%) on TUG-Kv4.0. In a card-sorting proxy using two of my comprehensive finals (60 items), its categories reflect solution method rather than surface features. Solving those same exams, it attains 27/30 and 25/30, with most misses in ruler-based ray tracing and circuit interpretation. On epistemology, five independent CLASS runs yield 100\% favorable, indicating a simulated expert-like stance. Framed as a "boiling frog" problem, the paper argues for a decisive jump: retire credit-bearing unsupervised closed-response online assessments; grade process evidence; use paper, whiteboarding; shift weight to modeling, data, and authentic labs; require transparent, citable AI use; rebuild problem types; and lean on research-based instruction and peer learning. The opportunity is to foreground what AI cannot substitute for: modeling the world, arguing from evidence, and making principled approximations.
- Open Access
When AI Evaluates Its Own Work: Validating Learner-Initiated, AI-Generated Physics Practice Problems
Large language models (LLMs) can now generate physics practice problems in real time, yet the educational value of these items hinges on rapid, reliable post-generation vetting. We investigated which automated checks are both technically feasible and pedagogically meaningful when exercises are produced on demand within a chatbot interface. A cohort of 34 introductory-physics students generated and attempted 543 problems during exam preparation. Each item was labeled by an expert on a wide range of quality attributes and presented to the learners in pairs to record their preference. We then (i) benchmarked three commodity LLMs as "judges" against the expert labels, (ii) quantified which attributes predict student choice via random-forest models, and (iii) triangulated these results with free-form exit surveys. Only a small subset of the original rubric proved necessary to reliably address student preferences either directly or by proxy. The study demonstrates that scalable formative assessment does not require exhaustive scoring: a carefully curated core of structural and learner-visible checks is sufficient to ensure both technical soundness and user appeal. The findings provide a practical blueprint for deploying real-time, AI-generated practice in physics and other quantitative disciplines.
Open AccessWhen AI Evaluates Its Own Work: Validating Learner-Initiated, AI-Generated Physics Practice Problems
Large language models (LLMs) can now generate physics practice problems in real time, yet the educational value of these items hinges on rapid, reliable post-generation vetting. We investigated which automated checks are both technically feasible and pedagogically meaningful when exercises are produced on demand within a chatbot interface. A cohort of 34 introductory-physics students generated and attempted 543 problems during exam preparation. Each item was labeled by an expert on a wide range of quality attributes and presented to the learners in pairs to record their preference. We then (i) benchmarked three commodity LLMs as "judges" against the expert labels, (ii) quantified which attributes predict student choice via random-forest models, and (iii) triangulated these results with free-form exit surveys. Only a small subset of the original rubric proved necessary to reliably address student preferences either directly or by proxy. The study demonstrates that scalable formative assessment does not require exhaustive scoring: a carefully curated core of structural and learner-visible checks is sufficient to ensure both technical soundness and user appeal. The findings provide a practical blueprint for deploying real-time, AI-generated practice in physics and other quantitative disciplines.
- Top Cited PapersOpen Access
- 8 July 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (2)
Multilingual performance of a multimodal artificial intelligence system on multisubject physics concept inventories
We investigate the multilingual and multimodal performance of a large language model-based artificial intelligence (AI) system, GPT-4o, using a diverse set of physics concept inventories spanning multiple languages and subject categories. The inventories, sourced from the PhysPort website, cover classical physics topics such as mechanics, electromagnetism, optics, and thermodynamics, as well as relativity, quantum mechanics, astronomy, mathematics, and laboratory skills. Unlike previous text-only studies, we uploaded the inventories as images to reflect what a student would see on paper, thereby assessing the system’s multimodal functionality. Our results indicate variation in performance across subjects, with laboratory skills standing out as the weakest. We also observe differences across languages, with English and European languages showing the strongest performance. Notably, the relative difficulty of an inventory item is largely independent of the language of the test. When comparing AI results to existing literature on student performance, we find that the AI system outperforms average postinstruction undergraduate students in all subject categories except laboratory skills. Furthermore, the AI performs worse on items requiring visual interpretation of images than on those that are purely text-based. While our exploratory findings show GPT-4o’s potential usefulness in physics education, they highlight the critical need for instructors to foster students’ ability to critically evaluate AI outputs, adapt curricula thoughtfully in response to AI advancements, and address equity concerns associated with AI integration.
- 8 July 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (2)
Top Cited PapersOpen AccessMultilingual performance of a multimodal artificial intelligence system on multisubject physics concept inventories
We investigate the multilingual and multimodal performance of a large language model-based artificial intelligence (AI) system, GPT-4o, using a diverse set of physics concept inventories spanning multiple languages and subject categories. The inventories, sourced from the PhysPort website, cover classical physics topics such as mechanics, electromagnetism, optics, and thermodynamics, as well as relativity, quantum mechanics, astronomy, mathematics, and laboratory skills. Unlike previous text-only studies, we uploaded the inventories as images to reflect what a student would see on paper, thereby assessing the system’s multimodal functionality. Our results indicate variation in performance across subjects, with laboratory skills standing out as the weakest. We also observe differences across languages, with English and European languages showing the strongest performance. Notably, the relative difficulty of an inventory item is largely independent of the language of the test. When comparing AI results to existing literature on student performance, we find that the AI system outperforms average postinstruction undergraduate students in all subject categories except laboratory skills. Furthermore, the AI performs worse on items requiring visual interpretation of images than on those that are purely text-based. While our exploratory findings show GPT-4o’s potential usefulness in physics education, they highlight the critical need for instructors to foster students’ ability to critically evaluate AI outputs, adapt curricula thoughtfully in response to AI advancements, and address equity concerns associated with AI integration.
- Top Cited PapersOpen Access
- 30 April 2025
- Published by Springer Nature in Discover Education
- Vol. 4 (1) , 1-15
Attending lectures in person, hybrid or online at a technical university: how do students choose after the pandemic, and what about the outcome?
In the aftermath of the COVID-19 pandemic, students have more choice of how to attend courses than ever before; for a large number of courses at a technical university, they are still able to watch the lectures live online or in recorded format later. We found that interactivity may bring back students into the classroom almost as effectively as withholding alternatives. However, we also found that these attendance modes by themselves have little influence on exam grades. Instead, exam outcomes are more strongly associated with students attitudes and interest. We also found indicators that focussing on live lectures during the running semester (on-campus or online) may be associated with better exam outcomes than intensive preparation phases before exams.
- 30 April 2025
- Published by Springer Nature in Discover Education
- Vol. 4 (1) , 1-15
Top Cited PapersOpen AccessAttending lectures in person, hybrid or online at a technical university: how do students choose after the pandemic, and what about the outcome?
In the aftermath of the COVID-19 pandemic, students have more choice of how to attend courses than ever before; for a large number of courses at a technical university, they are still able to watch the lectures live online or in recorded format later. We found that interactivity may bring back students into the classroom almost as effectively as withholding alternatives. However, we also found that these attendance modes by themselves have little influence on exam grades. Instead, exam outcomes are more strongly associated with students attitudes and interest. We also found indicators that focussing on live lectures during the running semester (on-campus or online) may be associated with better exam outcomes than intensive preparation phases before exams.
- Open Access
- 7 April 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (1)
Assessing confidence in AI-assisted grading of physics exams through psychometrics: An exploratory study
This study explores the use of artificial intelligence in grading high-stakes physics exams, emphasizing the application of psychometric methods, particularly item response theory, to evaluate the reliability of AI-assisted grading. We examine how grading rubrics can be iteratively refined and how threshold parameters can determine when AI-generated grades are reliable versus when human intervention is necessary. By adjusting thresholds for correctness measures and uncertainty, AI can grade with high precision, significantly reducing grading workloads while maintaining accuracy. Our findings show that AI can achieve a coefficient of determination of R 2 ≈ 0.91 when handling half of the grading load, and R 2 ≈ 0.96 for one-fifth of the load. These results demonstrate AI’s potential to assist in grading large-scale assessments, reducing both human effort and associated costs. However, the study underscores the importance of human oversight in cases of uncertainty or complex problem solving, ensuring the integrity of the grading process.
- 7 April 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (1)
Open AccessAssessing confidence in AI-assisted grading of physics exams through psychometrics: An exploratory study
This study explores the use of artificial intelligence in grading high-stakes physics exams, emphasizing the application of psychometric methods, particularly item response theory, to evaluate the reliability of AI-assisted grading. We examine how grading rubrics can be iteratively refined and how threshold parameters can determine when AI-generated grades are reliable versus when human intervention is necessary. By adjusting thresholds for correctness measures and uncertainty, AI can grade with high precision, significantly reducing grading workloads while maintaining accuracy. Our findings show that AI can achieve a coefficient of determination of R 2 ≈ 0.91 when handling half of the grading load, and R 2 ≈ 0.96 for one-fifth of the load. These results demonstrate AI’s potential to assist in grading large-scale assessments, reducing both human effort and associated costs. However, the study underscores the importance of human oversight in cases of uncertainty or complex problem solving, ensuring the integrity of the grading process.