Fabian Kieser
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Fabian Kieser has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Peter Wulff
Ludwigsburg University of Education
3 shared publications - Holger Maus
Department of Physics Education, Leibniz Institute for Science and Mathematics Education, Kiel, Germany
2 shared publications - Paul Tschisgale
Department of Physics Education, Leibniz Institute for Science and Mathematics Education, Kiel, Germany
2 shared publications - Stefan Petersen
Department of Physics Education, Leibniz Institute for Science and Mathematics Education, Kiel, Germany
2 shared publications - Ben Kroehs
Ruprecht Karl University of Heidelberg
1 shared publication - Jochen Kuhn
LMU Munich, Munich, Germany
1 shared publication
Latest Publications
- 1 January 2026
- Published by World Scientific Pub Co Pte Ltd in The Physics Educator
- Vol. 08
Developing an LLM-Based Feedback System Grounded in Evidence-Centered Design to Support Physics Problem Solving
Generative AI offers new opportunities for individualized and adaptive learning, e.g. through large language model (LLM)-based feedback systems. While LLMs can produce factually correct feedback for relatively straightforward conceptual tasks, delivering high-quality feedback for tasks that require advanced domain expertise — such as physics problem solving — remains a substantial challenge. This study presents the design and implementation of an LLM-based feedback system for physics problem solving grounded in evidence-centered design and reports a first evaluation within the German Physics Olympiad. Participants rated the usefulness and correctness of the generated feedback for each implemented problem. The collected ratings indicate that the feedback was generally perceived as useful and highly correct. However, an in-depth analysis revealed that the feedback contained errors in 20% of cases — errors that often went unnoticed by the students. We discuss the risks associated with uncritical reliance on LLM-based feedback and outline potential directions for generating more adaptive and reliable LLM-based feedback in the future.
- 1 January 2026
- Published by World Scientific Pub Co Pte Ltd in The Physics Educator
- Vol. 08
Developing an LLM-Based Feedback System Grounded in Evidence-Centered Design to Support Physics Problem Solving
Generative AI offers new opportunities for individualized and adaptive learning, e.g. through large language model (LLM)-based feedback systems. While LLMs can produce factually correct feedback for relatively straightforward conceptual tasks, delivering high-quality feedback for tasks that require advanced domain expertise — such as physics problem solving — remains a substantial challenge. This study presents the design and implementation of an LLM-based feedback system for physics problem solving grounded in evidence-centered design and reports a first evaluation within the German Physics Olympiad. Participants rated the usefulness and correctness of the generated feedback for each implemented problem. The collected ratings indicate that the feedback was generally perceived as useful and highly correct. However, an in-depth analysis revealed that the feedback contained errors in 20% of cases — errors that often went unnoticed by the students. We discuss the risks associated with uncritical reliance on LLM-based feedback and outline potential directions for generating more adaptive and reliable LLM-based feedback in the future.
- Top Cited PapersOpen Access
- 13 August 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (2)
Evaluating GPT- and reasoning-based large language models on Physics Olympiad problems: Surpassing human performance and implications for educational assessment
Large language models (LLMs) are now widely accessible, reaching learners across all educational levels. This development has raised concerns that their use may circumvent essential learning processes and compromise the integrity of established assessment formats. In physics education, where problem solving plays a central role in both instruction and assessment, it is therefore essential to understand the physics-specific problem-solving capabilities of LLMs. Such understanding is key to informing responsible and pedagogically sound approaches to integrating LLMs into instruction and assessment. This study therefore compares the problem-solving performance of a general-purpose LLM ( G P T − 4 o , using varying prompting techniques) and a reasoning-optimized model ( o 1 ) with that of participants in the German Physics Olympiad, based on a set of well-defined Olympiad problems. In addition to evaluating the correctness of the generated solutions, the study analyzes the characteristic strengths and limitations of LLM-generated solutions. The results of this study indicate that both tested LLMs ( G P T − 4 o and o 1 ) demonstrate advanced problem-solving capabilities on Olympiad-type physics problems, on average outperforming the human participants. Prompting techniques had little effect on G P T − 4 o ’s performance, and o 1 almost consistently outperformed both G P T − 4 o and the human benchmark. The main implications of these findings are twofold: LLMs pose a challenge for summative assessment in unsupervised settings, as they can solve advanced physics problems at a level that exceeds top-performing students, making it difficult to ensure the authenticity of student work. At the same time, their problem-solving capabilities offer potential for formative assessment, where LLMs can support students in evaluating their own solutions to problems.
- 13 August 2025
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 21 (2)
Top Cited PapersOpen AccessEvaluating GPT- and reasoning-based large language models on Physics Olympiad problems: Surpassing human performance and implications for educational assessment
Large language models (LLMs) are now widely accessible, reaching learners across all educational levels. This development has raised concerns that their use may circumvent essential learning processes and compromise the integrity of established assessment formats. In physics education, where problem solving plays a central role in both instruction and assessment, it is therefore essential to understand the physics-specific problem-solving capabilities of LLMs. Such understanding is key to informing responsible and pedagogically sound approaches to integrating LLMs into instruction and assessment. This study therefore compares the problem-solving performance of a general-purpose LLM ( G P T − 4 o , using varying prompting techniques) and a reasoning-optimized model ( o 1 ) with that of participants in the German Physics Olympiad, based on a set of well-defined Olympiad problems. In addition to evaluating the correctness of the generated solutions, the study analyzes the characteristic strengths and limitations of LLM-generated solutions. The results of this study indicate that both tested LLMs ( G P T − 4 o and o 1 ) demonstrate advanced problem-solving capabilities on Olympiad-type physics problems, on average outperforming the human participants. Prompting techniques had little effect on G P T − 4 o ’s performance, and o 1 almost consistently outperformed both G P T − 4 o and the human benchmark. The main implications of these findings are twofold: LLMs pose a challenge for summative assessment in unsupervised settings, as they can solve advanced physics problems at a level that exceeds top-performing students, making it difficult to ensure the authenticity of student work. At the same time, their problem-solving capabilities offer potential for formative assessment, where LLMs can support students in evaluating their own solutions to problems.
- Top Cited PapersOpen Access
- 25 October 2023
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 19 (2) , 020150
Educational data augmentation in physics education research using ChatGPT
Generative AI technologies such as large language models show novel potential to enhance educational research. For example, generative large language models were shown to be capable of solving quantitative reasoning tasks in physics and concept tests such as the Force Concept Inventory (FCI). Given the importance of such concept inventories for physics education research, and the challenges in developing them such as field testing with representative populations, this study seeks to examine to what extent a generative large language model could be utilized to generate a synthetic dataset for the FCI that exhibits content-related variability in responses. We use the recently introduced ChatGPT based on the GPT 4 generative large language model and investigate to what extent ChatGPT could solve the FCI accurately (RQ1) and could be prompted to solve the FCI as if it were a student belonging to a different cohort (RQ2). Furthermore, we study, to what extent ChatGPT could be prompted to solve the FCI as if it were a student having a different force- and mechanics-related preconception (RQ3). In alignment with other research, we found that ChatGPT could accurately solve the FCI. We furthermore found that prompting ChatGPT to respond to the inventory as if it belonged to a different cohort yielded no variance in responses, however, responding as if it had a certain preconception introduced much variance in responses that approximate real human responses on the FCI in some regards.
- 25 October 2023
- Published by American Physical Society (APS) in Physical Review Physics Education Research
- Vol. 19 (2) , 020150
Top Cited PapersOpen AccessEducational data augmentation in physics education research using ChatGPT
Generative AI technologies such as large language models show novel potential to enhance educational research. For example, generative large language models were shown to be capable of solving quantitative reasoning tasks in physics and concept tests such as the Force Concept Inventory (FCI). Given the importance of such concept inventories for physics education research, and the challenges in developing them such as field testing with representative populations, this study seeks to examine to what extent a generative large language model could be utilized to generate a synthetic dataset for the FCI that exhibits content-related variability in responses. We use the recently introduced ChatGPT based on the GPT 4 generative large language model and investigate to what extent ChatGPT could solve the FCI accurately (RQ1) and could be prompted to solve the FCI as if it were a student belonging to a different cohort (RQ2). Furthermore, we study, to what extent ChatGPT could be prompted to solve the FCI as if it were a student having a different force- and mechanics-related preconception (RQ3). In alignment with other research, we found that ChatGPT could accurately solve the FCI. We furthermore found that prompting ChatGPT to respond to the inventory as if it belonged to a different cohort yielded no variance in responses, however, responding as if it had a certain preconception introduced much variance in responses that approximate real human responses on the FCI in some regards.