Manfred Herrmann
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Manfred Herrmann has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Annette Wiegand
Department of Preventive Dentistry, Periodontology and Cariology University Medical Centre Göttingen Göttingen Germany
6 shared publications - Philipp Kanzow
Department of Restorative Dentistry, Periodontology and Endodontology University Medicine Greifswald Greifswald Germany
6 shared publications - Dennis Schmidt
Department of Preventive Dentistry, Periodontology and Cariology University Medical Center Göttingen Göttingen Germany
4 shared publications - Tobias Raupach
Institute of Medical Education, University Hospital Bonn, Bonn, Germany
4 shared publications - Torsten Wassmann
Department of Prosthodontics University Medical Center Göttingen Göttingen Germany
2 shared publications Claudia Tschammler
Department of Preventive Dentistry, Periodontology, and Cariology University Medical Center Göttingen Germany
1 shared publication
Latest Publications
- Open Access
- 27 June 2024
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 10 (1) , e58126
Use of Multiple-Choice Items in Summative Examinations: Questionnaire Survey Among German Undergraduate Dental Training Programs
Background: Multiple-choice examinations are frequently used in German dental schools. However, details regarding the used item types and applied scoring methods are lacking. Objective: This study aims to gain insight into the current use of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programs. Methods: A paper-based 10-item questionnaire regarding the used assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the deans of studies and to the heads of the Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using the Fisher exact test (P<.05). Results: The response rate amounted to 90% (27/30 dental schools). All respondent dental schools used multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (24/27, 89%), which accounted for the largest number of items in approximately half of the dental schools (13/27, 48%). Further item types (eg, conventional multiple-select items, Multiple-True-False, and Pick-N) were only used by fewer dental schools (≤67%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit and requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (14/19, 74% vs 4/8, 50%). However, this difference was statistically not significant (P=.38). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (15/27, 56%). Not a single school used alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (15/27, 56%). Conclusions: Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.
- 27 June 2024
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 10 (1) , e58126
Open AccessUse of Multiple-Choice Items in Summative Examinations: Questionnaire Survey Among German Undergraduate Dental Training Programs
Background: Multiple-choice examinations are frequently used in German dental schools. However, details regarding the used item types and applied scoring methods are lacking. Objective: This study aims to gain insight into the current use of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programs. Methods: A paper-based 10-item questionnaire regarding the used assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the deans of studies and to the heads of the Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using the Fisher exact test (P<.05). Results: The response rate amounted to 90% (27/30 dental schools). All respondent dental schools used multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (24/27, 89%), which accounted for the largest number of items in approximately half of the dental schools (13/27, 48%). Further item types (eg, conventional multiple-select items, Multiple-True-False, and Pick-N) were only used by fewer dental schools (≤67%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit and requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (14/19, 74% vs 4/8, 50%). However, this difference was statistically not significant (P=.38). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (15/27, 56%). Not a single school used alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (15/27, 56%). Conclusions: Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.
- Open Access
- 27 March 2023
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 9 (1) , e43792
Use of Multiple-Select Multiple-Choice Items in a Dental Undergraduate Curriculum: Retrospective Study Involving the Application of Different Scoring Methods
Background: Scoring and awarding credit are more complex for multiple-select items than for single-choice items. Forty-one different scoring methods were retrospectively applied to 2 multiple-select multiple-choice item types (Pick-N and Multiple-True-False [MTF]) from existing examination data. Objective: This study aimed to calculate and compare the mean scores for both item types by applying different scoring methods, and to investigate the effect of item quality on mean raw scores and the likelihood of resulting scores at or above the pass level (≥0.6). Methods: Items and responses from examinees (ie, marking events) were retrieved from previous examinations. Different scoring methods were retrospectively applied to the existing examination data to calculate corresponding examination scores. In addition, item quality was assessed using a validated checklist. Statistical analysis was performed using the Kruskal-Wallis test, Wilcoxon rank-sum test, and multiple logistic regression analysis (P<.05). Results: We analyzed 1931 marking events of 48 Pick-N items and 828 marking events of 18 MTF items. For both item types, scoring results widely differed between scoring methods (minimum: 0.02, maximum: 0.98; P<.001). Both the use of an inappropriate item type (34 items) and the presence of cues (30 items) impacted the scoring results. Inappropriately used Pick-N items resulted in lower mean raw scores (0.88 vs 0.93; P<.001), while inappropriately used MTF items resulted in higher mean raw scores (0.88 vs 0.85; P=.001). Mean raw scores were higher for MTF items with cues than for those without cues (0.91 vs 0.8; P<.001), while mean raw scores for Pick-N items with and without cues did not differ (0.89 vs 0.90; P=.09). Item quality also impacted the likelihood of resulting scores at or above the pass level (odds ratio ≤6.977). Conclusions: Educators should pay attention when using multiple-select multiple-choice items and select the most appropriate item type. Different item types, different scoring methods, and presence of cues are likely to impact examinees’ scores and overall examination results.
- 27 March 2023
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 9 (1) , e43792
Open AccessUse of Multiple-Select Multiple-Choice Items in a Dental Undergraduate Curriculum: Retrospective Study Involving the Application of Different Scoring Methods
Background: Scoring and awarding credit are more complex for multiple-select items than for single-choice items. Forty-one different scoring methods were retrospectively applied to 2 multiple-select multiple-choice item types (Pick-N and Multiple-True-False [MTF]) from existing examination data. Objective: This study aimed to calculate and compare the mean scores for both item types by applying different scoring methods, and to investigate the effect of item quality on mean raw scores and the likelihood of resulting scores at or above the pass level (≥0.6). Methods: Items and responses from examinees (ie, marking events) were retrieved from previous examinations. Different scoring methods were retrospectively applied to the existing examination data to calculate corresponding examination scores. In addition, item quality was assessed using a validated checklist. Statistical analysis was performed using the Kruskal-Wallis test, Wilcoxon rank-sum test, and multiple logistic regression analysis (P<.05). Results: We analyzed 1931 marking events of 48 Pick-N items and 828 marking events of 18 MTF items. For both item types, scoring results widely differed between scoring methods (minimum: 0.02, maximum: 0.98; P<.001). Both the use of an inappropriate item type (34 items) and the presence of cues (30 items) impacted the scoring results. Inappropriately used Pick-N items resulted in lower mean raw scores (0.88 vs 0.93; P<.001), while inappropriately used MTF items resulted in higher mean raw scores (0.88 vs 0.85; P=.001). Mean raw scores were higher for MTF items with cues than for those without cues (0.91 vs 0.8; P<.001), while mean raw scores for Pick-N items with and without cues did not differ (0.89 vs 0.90; P=.09). Item quality also impacted the likelihood of resulting scores at or above the pass level (odds ratio ≤6.977). Conclusions: Educators should pay attention when using multiple-select multiple-choice items and select the most appropriate item type. Different item types, different scoring methods, and presence of cues are likely to impact examinees’ scores and overall examination results.
- Open Access
- 25 October 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Use of Multiple-Select Multiple-Choice Items in a Dental Undergraduate Curriculum: Retrospective Study Involving the Application of Different Scoring Methods (Preprint)
BACKGROUND Scoring and awarding credit are more complex for multiple-select items than for single-choice items. Forty-one different scoring methods were retrospectively applied to 2 multiple-select multiple-choice item types (Pick-N and Multiple-True-False [MTF]) from existing examination data. OBJECTIVE This study aimed to calculate and compare the mean scores for both item types by applying different scoring methods, and to investigate the effect of item quality on mean raw scores and the likelihood of resulting scores at or above the pass level (≥0.6). METHODS Items and responses from examinees (ie, marking events) were retrieved from previous examinations. Different scoring methods were retrospectively applied to the existing examination data to calculate corresponding examination scores. In addition, item quality was assessed using a validated checklist. Statistical analysis was performed using the Kruskal-Wallis test, Wilcoxon rank-sum test, and multiple logistic regression analysis (P<.05). RESULTS We analyzed 1931 marking events of 48 Pick-N items and 828 marking events of 18 MTF items. For both item types, scoring results widely differed between scoring methods (minimum: 0.02, maximum: 0.98; P<.001). Both the use of an inappropriate item type (34 items) and the presence of cues (30 items) impacted the scoring results. Inappropriately used Pick-N items resulted in lower mean raw scores (0.88 vs 0.93; P<.001), while inappropriately used MTF items resulted in higher mean raw scores (0.88 vs 0.85; P=.001). Mean raw scores were higher for MTF items with cues than for those without cues (0.91 vs 0.8; P<.001), while mean raw scores for Pick-N items with and without cues did not differ (0.89 vs 0.90; P=.09). Item quality also impacted the likelihood of resulting scores at or above the pass level (odds ratio ≤6.977). CONCLUSIONS Educators should pay attention when using multiple-select multiple-choice items and select the most appropriate item type. Different item types, different scoring methods, and presence of cues are likely to impact examinees’ scores and overall examination results.
- 25 October 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessUse of Multiple-Select Multiple-Choice Items in a Dental Undergraduate Curriculum: Retrospective Study Involving the Application of Different Scoring Methods (Preprint)
BACKGROUND Scoring and awarding credit are more complex for multiple-select items than for single-choice items. Forty-one different scoring methods were retrospectively applied to 2 multiple-select multiple-choice item types (Pick-N and Multiple-True-False [MTF]) from existing examination data. OBJECTIVE This study aimed to calculate and compare the mean scores for both item types by applying different scoring methods, and to investigate the effect of item quality on mean raw scores and the likelihood of resulting scores at or above the pass level (≥0.6). METHODS Items and responses from examinees (ie, marking events) were retrieved from previous examinations. Different scoring methods were retrospectively applied to the existing examination data to calculate corresponding examination scores. In addition, item quality was assessed using a validated checklist. Statistical analysis was performed using the Kruskal-Wallis test, Wilcoxon rank-sum test, and multiple logistic regression analysis (P<.05). RESULTS We analyzed 1931 marking events of 48 Pick-N items and 828 marking events of 18 MTF items. For both item types, scoring results widely differed between scoring methods (minimum: 0.02, maximum: 0.98; P<.001). Both the use of an inappropriate item type (34 items) and the presence of cues (30 items) impacted the scoring results. Inappropriately used Pick-N items resulted in lower mean raw scores (0.88 vs 0.93; P<.001), while inappropriately used MTF items resulted in higher mean raw scores (0.88 vs 0.85; P=.001). Mean raw scores were higher for MTF items with cues than for those without cues (0.91 vs 0.8; P<.001), while mean raw scores for Pick-N items with and without cues did not differ (0.89 vs 0.90; P=.09). Item quality also impacted the likelihood of resulting scores at or above the pass level (odds ratio ≤6.977). CONCLUSIONS Educators should pay attention when using multiple-select multiple-choice items and select the most appropriate item type. Different item types, different scoring methods, and presence of cues are likely to impact examinees’ scores and overall examination results.
- 13 September 2022
- Published by Elsevier in Educational Research Review
- Vol. 37
Relation between examinees’ true knowledge and examination scores: systematic review and exemplary calculations on Pick-N items
- 13 September 2022
- Published by Elsevier in Educational Research Review
- Vol. 37
Relation between examinees’ true knowledge and examination scores: systematic review and exemplary calculations on Pick-N items
- 1 September 2018
- Published by Wiley in Journal of Dental Education
- Vol. 82 (9) , 943-948
Dental Students’ Factual and Procedural Knowledge Retention in Operative Dentistry in a Clinical Dental Curriculum
The aim of this study was to evaluate dental students’ retention of factual and procedural knowledge gained in a preclinical course in operative dentistry during a clinical dental curriculum. In 2017, all 157 seventh- to tenth-semester dental students at a dental school in Germany were asked to repeat the same written examination performed at the end of the preclinical course in the sixth semester. The examinations consisted of 30 multiple-choice questions covering factual and procedural knowledge. The percentage of correctly answered questions per exam and differences in correct answers between the original examination and the re-examination (per question) were analyzed. Students were also asked to self-rate their percentage of correctly answered questions and their knowledge in various disciplines of operative dentistry at the times of the original examination and the re-examination. After exclusions, data were analyzed for 129 students, for a participation rate of 82%. For the seventh- and tenth-semester students, the results on the original examination and the re-examination were not significantly different, while the eighth- and ninth-semester students performed significantly better on the original examination than the re-examination. In all semesters, procedural knowledge remained stable between the original examination and the re-examination, while factual knowledge decreased slightly. Their performance on the original examination was underestimated by the eighth- and ninth-semester students. All the students underestimated their performance on the re-examination. Students mostly rated their knowledge level significantly higher on the original examination than on the re-examination. Overall, this study found that factual and procedural knowledge gained in a preclinical course in operative dentistry was not increased during the clinical dental curriculum.
- 1 September 2018
- Published by Wiley in Journal of Dental Education
- Vol. 82 (9) , 943-948
Dental Students’ Factual and Procedural Knowledge Retention in Operative Dentistry in a Clinical Dental Curriculum
The aim of this study was to evaluate dental students’ retention of factual and procedural knowledge gained in a preclinical course in operative dentistry during a clinical dental curriculum. In 2017, all 157 seventh- to tenth-semester dental students at a dental school in Germany were asked to repeat the same written examination performed at the end of the preclinical course in the sixth semester. The examinations consisted of 30 multiple-choice questions covering factual and procedural knowledge. The percentage of correctly answered questions per exam and differences in correct answers between the original examination and the re-examination (per question) were analyzed. Students were also asked to self-rate their percentage of correctly answered questions and their knowledge in various disciplines of operative dentistry at the times of the original examination and the re-examination. After exclusions, data were analyzed for 129 students, for a participation rate of 82%. For the seventh- and tenth-semester students, the results on the original examination and the re-examination were not significantly different, while the eighth- and ninth-semester students performed significantly better on the original examination than the re-examination. In all semesters, procedural knowledge remained stable between the original examination and the re-examination, while factual knowledge decreased slightly. Their performance on the original examination was underestimated by the eighth- and ninth-semester students. All the students underestimated their performance on the re-examination. Students mostly rated their knowledge level significantly higher on the original examination than on the re-examination. Overall, this study found that factual and procedural knowledge gained in a preclinical course in operative dentistry was not increased during the clinical dental curriculum.