Sonja Breuer
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sonja Breuer has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Tuulia M. Ortner
Paris Lodron Universität Salzburg
3 shared publications Thomas Scherndl
Division of Psychological Assessment, Department of Psychology, Paris Lodron University, Salzburg, Austria
2 shared publications- Thomas Scherndl
Department of Psychology, University of Salzburg, Austria
1 shared publication Tuulia M. Ortner
Division of Psychological Assessment, Department of Psychology, Paris Lodron University, Salzburg, Austria
1 shared publication
Latest Publications
- 5 February 2025
- Published by Springer Nature
- p. 697-708
Eignungsdiagnostische Interviews
- Open Access
- 3 May 2023
- Published by The Royal Society in Royal Society Open Science
- Vol. 10 (5) , 220456
Effects of response format on achievement and aptitude assessment results: multi-level random effects meta-analyses
Psychological achievement and aptitude tests are fundamental elements of the everyday school, academic and professional lives of students, instructors, job applicants, researchers and policymakers. In line with growing demands for fair psychological assessment tools, we aimed to identify psychometric features of tests, test situations and test-taker characteristics that may contribute to the emergence of test bias. Multi-level random effects meta-analyses were conducted to estimate mean effect sizes for differences and relations between scores from achievement or aptitude measures with open-ended (OE) versus closed-ended (CE) response formats. Results from 102 primary studies with 392 effect sizes revealed positive relations between CE and OE assessments (mean r = 0.67, 95% CI [0.57; 0.76]), with negative pooled effect sizes for the difference between the two response formats (mean d av = −0.65; 95% CI [−0.78; −0.53]). Significantly higher scores were obtained on CE exams. Stem-equivalency of items, low-stakes test situations, written short answer OE question types, studies conducted outside the United States and before the year 2000, and test-takers' achievement motivation and sex were at least partially associated with smaller differences and/or larger relations between scores from OE and CE formats. Limitations and the results’ implications for practitioners in achievement and aptitude testing are discussed.
- 3 May 2023
- Published by The Royal Society in Royal Society Open Science
- Vol. 10 (5) , 220456
Open AccessEffects of response format on achievement and aptitude assessment results: multi-level random effects meta-analyses
Psychological achievement and aptitude tests are fundamental elements of the everyday school, academic and professional lives of students, instructors, job applicants, researchers and policymakers. In line with growing demands for fair psychological assessment tools, we aimed to identify psychometric features of tests, test situations and test-taker characteristics that may contribute to the emergence of test bias. Multi-level random effects meta-analyses were conducted to estimate mean effect sizes for differences and relations between scores from achievement or aptitude measures with open-ended (OE) versus closed-ended (CE) response formats. Results from 102 primary studies with 392 effect sizes revealed positive relations between CE and OE assessments (mean r = 0.67, 95% CI [0.57; 0.76]), with negative pooled effect sizes for the difference between the two response formats (mean d av = −0.65; 95% CI [−0.78; −0.53]). Significantly higher scores were obtained on CE exams. Stem-equivalency of items, low-stakes test situations, written short answer OE question types, studies conducted outside the United States and before the year 2000, and test-takers' achievement motivation and sex were at least partially associated with smaller differences and/or larger relations between scores from OE and CE formats. Limitations and the results’ implications for practitioners in achievement and aptitude testing are discussed.
- Open Access
- 1 April 2023
- Published by Hogrefe Publishing Group in Journal of Individual Differences
- Vol. 44 (2) , 83-96
Find the Mistake!
Reasoning ability has commonly been regarded as the best predictor of academic and occupational success. Due to concerns about the validity of multiple-choice (MC) formats, test security breaches, and the fact that the difficulty levels of most existing reasoning assessments target the population mean, there is a constant need for new reliable and valid test instruments that can be applied to assess fluid intelligence in advanced cognitive performance areas. We developed a novel computerized figural matrices test to assess nonverbal reasoning for university student aptitude assessment. In two studies, we generated, revised, and empirically validated the Isometric Matrices Test (IMT). Our results show that the IMT is less prone to test-wiseness strategies than existing reasoning tests. In a third study, we created and evaluated an innovative Find the Mistake (FtM) response format as an alternative to classical multiple-choice formats. Overall, both response formats revealed satisfactory psychometric quality in terms of item difficulties and discrimination, test-retest reliability, construct and criterion validity, and Rasch or two-parameter logistic (2PL) model fit, but in one MC version, the internal consistency was low due to negative discrimination indices. The MC response format turned out to be easier than the FtM format, with men slightly outperforming women in both response modes. We propose the IMT as a useful tool for assessing nonverbal reasoning ability in above-average performance areas and discuss the automatic generation of larger IMT item pools for adaptive testing in order to increase test security and reliability.
- 1 April 2023
- Published by Hogrefe Publishing Group in Journal of Individual Differences
- Vol. 44 (2) , 83-96
Open AccessFind the Mistake!
Reasoning ability has commonly been regarded as the best predictor of academic and occupational success. Due to concerns about the validity of multiple-choice (MC) formats, test security breaches, and the fact that the difficulty levels of most existing reasoning assessments target the population mean, there is a constant need for new reliable and valid test instruments that can be applied to assess fluid intelligence in advanced cognitive performance areas. We developed a novel computerized figural matrices test to assess nonverbal reasoning for university student aptitude assessment. In two studies, we generated, revised, and empirically validated the Isometric Matrices Test (IMT). Our results show that the IMT is less prone to test-wiseness strategies than existing reasoning tests. In a third study, we created and evaluated an innovative Find the Mistake (FtM) response format as an alternative to classical multiple-choice formats. Overall, both response formats revealed satisfactory psychometric quality in terms of item difficulties and discrimination, test-retest reliability, construct and criterion validity, and Rasch or two-parameter logistic (2PL) model fit, but in one MC version, the internal consistency was low due to negative discrimination indices. The MC response format turned out to be easier than the FtM format, with men slightly outperforming women in both response modes. We propose the IMT as a useful tool for assessing nonverbal reasoning ability in above-average performance areas and discuss the automatic generation of larger IMT item pools for adaptive testing in order to increase test security and reliability.
- Open Access
- 20 February 2020
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 5
Effects of Response Format on Psychometric Properties and Fairness of a Matrices Test: Multiple Choice vs. Free Response
Reasoning is regarded to be an essential facet of fundamental cognitive abilities. As examinee characteristics may affect performance in Reasoning tests, concern about maintaining fairness is expressed. The purpose of the current study was to examine effects of response format on psychometric properties and fairness of a matrices test according to examinee's sex, risk propensity, and test anxiety. A total of 433 German-speaking pupils (aged 14–20) were randomly assigned to either a multiple choice or a free response version of the same 25-item test. Data analysis yielded Rasch-homogeneous 23-item versions, with higher reliability, but lower criterion validity for the free response test. No interactions between response format and sex, test anxiety, or risk propensity were revealed, but a significant main effect of sex: men out-performed women in reasoning irrespective of response format. Results are discussed with reference to attributes of the test situation and sample characteristics.
- 20 February 2020
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 5
Open AccessEffects of Response Format on Psychometric Properties and Fairness of a Matrices Test: Multiple Choice vs. Free Response
Reasoning is regarded to be an essential facet of fundamental cognitive abilities. As examinee characteristics may affect performance in Reasoning tests, concern about maintaining fairness is expressed. The purpose of the current study was to examine effects of response format on psychometric properties and fairness of a matrices test according to examinee's sex, risk propensity, and test anxiety. A total of 433 German-speaking pupils (aged 14–20) were randomly assigned to either a multiple choice or a free response version of the same 25-item test. Data analysis yielded Rasch-homogeneous 23-item versions, with higher reliability, but lower criterion validity for the free response test. No interactions between response format and sex, test anxiety, or risk propensity were revealed, but a significant main effect of sex: men out-performed women in reasoning irrespective of response format. Results are discussed with reference to attributes of the test situation and sample characteristics.