Julian F. Lohmann
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Julian F. Lohmann has a total of 29 co-authors. Here are the top co-authors by number of shared publications:
- Jens Möller
Institute for Psychology of Learning and Instruction, Kiel University, Kiel, Germany
4 shared publications - Martin Hecht
Department of Psychology, Helmut Schmidt University, Hamburg, Germany
4 shared publications - Fynn Junge
Institute for Psychology of Learning and Instruction, Kiel University, Kiel, Germany
3 shared publications - Ruth Trüb
University of Applied Sciences and Arts Northwestern Switzerland, Basel, Switzerland
3 shared publications - Stefan Daniel Keller
Pädagogische Hochschule Zürich, Zürich, Schweiz
3 shared publications - Steffen Zitzmann
Department of Psychology, Medical School Hamburg
3 shared publications
Latest Publications
- Open Access
- 20 October 2026
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- p. 1-13
A One-Stage Approach for Estimating a Meta-Analytic Continuous-Time Random-Intercept Cross-Lagged Panel Model
- 20 October 2026
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- p. 1-13
Open AccessA One-Stage Approach for Estimating a Meta-Analytic Continuous-Time Random-Intercept Cross-Lagged Panel Model
- Open Access
- 7 October 2026
- Published by Elsevier in Applied Studies in Innovative Learning and Teaching
- Vol. 3
Color-based signaling of argumentative elements: An empirical analysis of its impact on the assessment of EFL argumentative essays
- 7 October 2026
- Published by Elsevier in Applied Studies in Innovative Learning and Teaching
- Vol. 3
Open AccessColor-based signaling of argumentative elements: An empirical analysis of its impact on the assessment of EFL argumentative essays
- Open Access
- 20 September 2026
- Published by Elsevier in SSRN Electronic Journal
Reducing Halo Effects in Elementary School EFL Writing Assessment: The Impact of Halo Prompts
Assessing texts from pupils learning English as a foreign language at elementary school poses various challenges, one being the distorting influence of orthography, vocabulary and punctuation on the assessment of other criteria. With a pre-post control group design, this study investigated whether the existence of these halo effects can be confirmed, and whether they can be reduced through repeated halo prompts. 284 pre-service teachers assessed six texts each, three in the pre-test and three in the post-test. The texts were randomly drawn from a larger text sample from a sixth-grade corpus. Participants were randomly assigned to one of three experimental groups (receiving halo prompts cautioning against orthography-, vocabulary- or punctuation-induced halo effects) or to the control group (neutral prompt). Multi-level regression analyses showed that orthography- and vocabulary-induced halo effects exist and that they could be significantly reduced by the interventions. Including halo prompts in the assessment process can therefore help to improve the quality of the assessment. The occurrence of punctuation-induced halo effects could not be confirmed. Teachers, teacher educators and language testing professionals should be aware that orthography and vocabulary can distort EFL writing assessment at elementary school level, and that awareness of this bias can reduce its impact.
- 20 September 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessReducing Halo Effects in Elementary School EFL Writing Assessment: The Impact of Halo Prompts
Assessing texts from pupils learning English as a foreign language at elementary school poses various challenges, one being the distorting influence of orthography, vocabulary and punctuation on the assessment of other criteria. With a pre-post control group design, this study investigated whether the existence of these halo effects can be confirmed, and whether they can be reduced through repeated halo prompts. 284 pre-service teachers assessed six texts each, three in the pre-test and three in the post-test. The texts were randomly drawn from a larger text sample from a sixth-grade corpus. Participants were randomly assigned to one of three experimental groups (receiving halo prompts cautioning against orthography-, vocabulary- or punctuation-induced halo effects) or to the control group (neutral prompt). Multi-level regression analyses showed that orthography- and vocabulary-induced halo effects exist and that they could be significantly reduced by the interventions. Including halo prompts in the assessment process can therefore help to improve the quality of the assessment. The occurrence of punctuation-induced halo effects could not be confirmed. Teachers, teacher educators and language testing professionals should be aware that orthography and vocabulary can distort EFL writing assessment at elementary school level, and that awareness of this bias can reduce its impact.
- Open Access
- 1 August 2026
- Published by American Psychological Association (APA) in Journal of Educational Psychology
- Vol. 118 (6) , 941-960
Testing teacher judgments comprehensively: Accuracy, halo, frame of reference, strategy, and personality effects in holistic and analytic assessments of student essays.
- 1 August 2026
- Published by American Psychological Association (APA) in Journal of Educational Psychology
- Vol. 118 (6) , 941-960
Open AccessTesting teacher judgments comprehensively: Accuracy, halo, frame of reference, strategy, and personality effects in holistic and analytic assessments of student essays.
- Open Access
- 30 April 2026
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 33 (5) , 1-15
Bayesian Hierarchical Moderated Factor Analysis for Testing Measurement Invariance in Multilevel Data: Model Development, Simulation Studies, and Experience Sampling Application
- 30 April 2026
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 33 (5) , 1-15
Open AccessBayesian Hierarchical Moderated Factor Analysis for Testing Measurement Invariance in Multilevel Data: Model Development, Simulation Studies, and Experience Sampling Application
- Open Access
- 25 March 2026
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 33 (4) , 1-21
Evaluating a Bayesian Approach for Estimating Moderator Effects in Parameter-Based Meta-Analytic Structural Equation Modeling
- 25 March 2026
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 33 (4) , 1-21
Open AccessEvaluating a Bayesian Approach for Estimating Moderator Effects in Parameter-Based Meta-Analytic Structural Equation Modeling
- Open Access
- 1 October 2025
- Published by American Psychological Association (APA) in Psychological Bulletin
- Vol. 151 (10) , 1280-1306
Data extraction by generative artificial intelligence: Assessing determinants of accuracy using human-extracted data from systematic review databases.
Psychological science requires reliable measures. Within systematic literature reviews, reliability hinges on high interrater agreement during data extraction. Yet, the extraction process has been time-consuming. Efforts to accelerate the process using technology have shown limited success until generative artificial intelligence (genAI), particularly large language models (LLMs), accurately extracted variables from medical studies. Nonetheless, for psychological researchers, it remains unclear how to utilize genAI for data extraction, given the range of tested variables, the medical context, and the variability in accuracy. We systematically assessed extraction accuracy and error patterns across domains in psychology by comparing genAI-extracted and human-extracted data from 22 systematic review databases published in the Psychological Bulletin. Eight LLMs extracted 312,329 data points from 2,179 studies on 186 variables. LLM extractions achieved unacceptable accuracy on all metrics for 20% of variables. For 46% of variables, accuracy was acceptable for some metrics and unacceptable for others. LLMs reached acceptable but not high accuracy on all metrics in 15%, high but not excellent in 8%, and excellent accuracy in 12% of variables. Accuracy varied most between variables, less between systematic reviews, and least between LLMs. Moderator analyses using a hierarchical logistic regression, hierarchical linear model, and meta-analysis revealed that accuracy was higher for variables describing studies' context and moderator variables compared to variables for effect size calculation. Also, accuracy was higher in systematic reviews with more detailed variable descriptions and positively correlated with model sizes. We discuss directions for investigating ways to use genAI to accelerate data extractions while ensuring meaningful human control. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
- 1 October 2025
- Published by American Psychological Association (APA) in Psychological Bulletin
- Vol. 151 (10) , 1280-1306
Open AccessData extraction by generative artificial intelligence: Assessing determinants of accuracy using human-extracted data from systematic review databases.
Psychological science requires reliable measures. Within systematic literature reviews, reliability hinges on high interrater agreement during data extraction. Yet, the extraction process has been time-consuming. Efforts to accelerate the process using technology have shown limited success until generative artificial intelligence (genAI), particularly large language models (LLMs), accurately extracted variables from medical studies. Nonetheless, for psychological researchers, it remains unclear how to utilize genAI for data extraction, given the range of tested variables, the medical context, and the variability in accuracy. We systematically assessed extraction accuracy and error patterns across domains in psychology by comparing genAI-extracted and human-extracted data from 22 systematic review databases published in the Psychological Bulletin. Eight LLMs extracted 312,329 data points from 2,179 studies on 186 variables. LLM extractions achieved unacceptable accuracy on all metrics for 20% of variables. For 46% of variables, accuracy was acceptable for some metrics and unacceptable for others. LLMs reached acceptable but not high accuracy on all metrics in 15%, high but not excellent in 8%, and excellent accuracy in 12% of variables. Accuracy varied most between variables, less between systematic reviews, and least between LLMs. Moderator analyses using a hierarchical logistic regression, hierarchical linear model, and meta-analysis revealed that accuracy was higher for variables describing studies' context and moderator variables compared to variables for effect size calculation. Also, accuracy was higher in systematic reviews with more detailed variable descriptions and positively correlated with model sizes. We discuss directions for investigating ways to use genAI to accelerate data extractions while ensuring meaningful human control. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
- Open Access
- 2 June 2025
- Published by Springer Nature in Educational Psychology Review
- Vol. 37 (2) , 1-45
A More Comprehensive, More Reliable Multilevel Approach for Assessing and Modeling Teacher Judgment Accuracy Using Latent Variables
We propose a novel approach for modeling judgment accuracy that, for the first time, allows for simultaneously considering the rank, level, and differentiation component, the predominantly applied operationalization of teacher judgment accuracy. These components are conceptualized as latent, unobserved individual abilities. The model is introduced technically and its functionality is illustrated. Next, several model extensions are described that enhance the model’s capabilities to address important research questions in teacher judgment accuracy research, such as concerning moderators of judgment accuracy or the effect of judgment accuracy on learning outcomes. We study the newly proposed model in three simulation studies demonstrating that our approach provides more accurate individual-level estimates of judgment accuracy components than the traditional, still widely applied person- and component-wise calculation (simulation study 1), that our approach yields more accurate standard errors of moderation effects of judgment accuracy components resulting in higher true positive rates across various typical sample size scenarios (simulation study 2), and that our approach yields lower parameter bias and higher coverage rates of predictive effects (simulation study 3). The findings underscore the model’s potential for improving the assessment and modeling of judgment accuracy although the improvement over the person-score-based two-step approach was rather small in some conditions. We present two real data examples, including a step-by-step tutorial how to apply the newly proposed approach, and conclude by discussing the implications of our results and suggesting directions for future research. Easy-to-use R code is provided to simplify the application of the new judgment accuracy model.
- 2 June 2025
- Published by Springer Nature in Educational Psychology Review
- Vol. 37 (2) , 1-45
Open AccessA More Comprehensive, More Reliable Multilevel Approach for Assessing and Modeling Teacher Judgment Accuracy Using Latent Variables
We propose a novel approach for modeling judgment accuracy that, for the first time, allows for simultaneously considering the rank, level, and differentiation component, the predominantly applied operationalization of teacher judgment accuracy. These components are conceptualized as latent, unobserved individual abilities. The model is introduced technically and its functionality is illustrated. Next, several model extensions are described that enhance the model’s capabilities to address important research questions in teacher judgment accuracy research, such as concerning moderators of judgment accuracy or the effect of judgment accuracy on learning outcomes. We study the newly proposed model in three simulation studies demonstrating that our approach provides more accurate individual-level estimates of judgment accuracy components than the traditional, still widely applied person- and component-wise calculation (simulation study 1), that our approach yields more accurate standard errors of moderation effects of judgment accuracy components resulting in higher true positive rates across various typical sample size scenarios (simulation study 2), and that our approach yields lower parameter bias and higher coverage rates of predictive effects (simulation study 3). The findings underscore the model’s potential for improving the assessment and modeling of judgment accuracy although the improvement over the person-score-based two-step approach was rather small in some conditions. We present two real data examples, including a step-by-step tutorial how to apply the newly proposed approach, and conclude by discussing the implications of our results and suggesting directions for future research. Easy-to-use R code is provided to simplify the application of the new judgment accuracy model.
- Open Access
- 13 September 2024
- Published by Elsevier in International Journal of Artificial Intelligence in Education
- Vol. 35 (3) , 1178-1217
Neural Networks or Linguistic Features? - Comparing Different Machine-Learning Approaches for Automated Assessment of Text Quality Traits Among L1- and L2-Learners’ Argumentative Essays
- 13 September 2024
- Published by Elsevier in International Journal of Artificial Intelligence in Education
- Vol. 35 (3) , 1178-1217
Open AccessNeural Networks or Linguistic Features? - Comparing Different Machine-Learning Approaches for Automated Assessment of Text Quality Traits Among L1- and L2-Learners’ Argumentative Essays
- 3 May 2023
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 31 (1) , 151-164
Studying Between-Subject Differences in Trends and Dynamics: Introducing the Random Coefficients Continuous-Time Latent Curve Model with Structured Residuals
The recently proposed continuous-time latent curve model with structured residuals (CT-LCM-SR) addresses several challenges associated with longitudinal data analysis in the behavioral sciences. First, it provides information about process trends and dynamics. Second, using the continuous-time framework, the CT-LCM-SR can handle unequally spaced measurement occasions and describes processes independently of the length of the time intervals used in a given study. Third, it is a hierarchical model. Thus, multiple subjects can be analyzed simultaneously. However, subjects might also differ in dynamics and trends. Therefore, in the present paper, we extend the CT-LCM-SR to capture these differences as well. This newly proposed random coefficients continuous-time latent curve model with structured residuals (RC-CT-LCM-SR) is introduced theoretically and technically. Additionally, we provide an illustrative example with data from the Health and Retirement Study (HRS), and we show how the RC-CT-LCM-SR can be used to study multiple sources of between-subject differences over time.
- 3 May 2023
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 31 (1) , 151-164
Studying Between-Subject Differences in Trends and Dynamics: Introducing the Random Coefficients Continuous-Time Latent Curve Model with Structured Residuals
The recently proposed continuous-time latent curve model with structured residuals (CT-LCM-SR) addresses several challenges associated with longitudinal data analysis in the behavioral sciences. First, it provides information about process trends and dynamics. Second, using the continuous-time framework, the CT-LCM-SR can handle unequally spaced measurement occasions and describes processes independently of the length of the time intervals used in a given study. Third, it is a hierarchical model. Thus, multiple subjects can be analyzed simultaneously. However, subjects might also differ in dynamics and trends. Therefore, in the present paper, we extend the CT-LCM-SR to capture these differences as well. This newly proposed random coefficients continuous-time latent curve model with structured residuals (RC-CT-LCM-SR) is introduced theoretically and technically. Additionally, we provide an illustrative example with data from the Health and Retirement Study (HRS), and we show how the RC-CT-LCM-SR can be used to study multiple sources of between-subject differences over time.