Ben Kroehs
Research Subjects
Ben Kroehs has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Fabian Kieser
Department of Physics Education Research, Free University of Berlin, Berlin, Germany
1 shared publication - Holger Maus
Department of Physics Education, Leibniz Institute for Science and Mathematics Education, Kiel, Germany
1 shared publication - Paul Tschisgale
Department of Physics Education, Leibniz Institute for Science and Mathematics Education, Kiel, Germany
1 shared publication - Peter Wulff
Ludwigsburg University of Education
1 shared publication - Stefan Petersen
Department of Physics Education, Leibniz Institute for Science and Mathematics Education, Kiel, Germany
1 shared publication
Latest Publications
- Top Cited PapersOpen Access
- 13 August 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (2)
Evaluating GPT- and reasoning-based large language models on Physics Olympiad problems: Surpassing human performance and implications for educational assessment
Large language models (LLMs) are now widely accessible, reaching learners across all educational levels. This development has raised concerns that their use may circumvent essential learning processes and compromise the integrity of established assessment formats. In physics education, where problem solving plays a central role in both instruction and assessment, it is therefore essential to understand the physics-specific problem-solving capabilities of LLMs. Such understanding is key to informing responsible and pedagogically sound approaches to integrating LLMs into instruction and assessment. This study therefore compares the problem-solving performance of a general-purpose LLM ( G P T − 4 o , using varying prompting techniques) and a reasoning-optimized model ( o 1 ) with that of participants in the German Physics Olympiad, based on a set of well-defined Olympiad problems. In addition to evaluating the correctness of the generated solutions, the study analyzes the characteristic strengths and limitations of LLM-generated solutions. The results of this study indicate that both tested LLMs ( G P T − 4 o and o 1 ) demonstrate advanced problem-solving capabilities on Olympiad-type physics problems, on average outperforming the human participants. Prompting techniques had little effect on G P T − 4 o ’s performance, and o 1 almost consistently outperformed both G P T − 4 o and the human benchmark. The main implications of these findings are twofold: LLMs pose a challenge for summative assessment in unsupervised settings, as they can solve advanced physics problems at a level that exceeds top-performing students, making it difficult to ensure the authenticity of student work. At the same time, their problem-solving capabilities offer potential for formative assessment, where LLMs can support students in evaluating their own solutions to problems.
- 13 August 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (2)
Top Cited PapersOpen AccessEvaluating GPT- and reasoning-based large language models on Physics Olympiad problems: Surpassing human performance and implications for educational assessment
Large language models (LLMs) are now widely accessible, reaching learners across all educational levels. This development has raised concerns that their use may circumvent essential learning processes and compromise the integrity of established assessment formats. In physics education, where problem solving plays a central role in both instruction and assessment, it is therefore essential to understand the physics-specific problem-solving capabilities of LLMs. Such understanding is key to informing responsible and pedagogically sound approaches to integrating LLMs into instruction and assessment. This study therefore compares the problem-solving performance of a general-purpose LLM ( G P T − 4 o , using varying prompting techniques) and a reasoning-optimized model ( o 1 ) with that of participants in the German Physics Olympiad, based on a set of well-defined Olympiad problems. In addition to evaluating the correctness of the generated solutions, the study analyzes the characteristic strengths and limitations of LLM-generated solutions. The results of this study indicate that both tested LLMs ( G P T − 4 o and o 1 ) demonstrate advanced problem-solving capabilities on Olympiad-type physics problems, on average outperforming the human participants. Prompting techniques had little effect on G P T − 4 o ’s performance, and o 1 almost consistently outperformed both G P T − 4 o and the human benchmark. The main implications of these findings are twofold: LLMs pose a challenge for summative assessment in unsupervised settings, as they can solve advanced physics problems at a level that exceeds top-performing students, making it difficult to ensure the authenticity of student work. At the same time, their problem-solving capabilities offer potential for formative assessment, where LLMs can support students in evaluating their own solutions to problems.