Alyssa A. Pradarelli
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Alyssa A. Pradarelli has a total of 60 co-authors. Here are the top co-authors by number of shared publications:
- Andrew E. Krumm
Department of Learning Health Sciences, University of Michigan, Ann Arbor, USA
6 shared publications - Brian C. George
Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, MI
5 shared publications - Kayla M. Marcotte
Department of Learning Health Sciences, University of Michigan, Ann Arbor, USA
4 shared publications - Rebecca Moreci
Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, MI
3 shared publications - Roy Phitayakorn
Department of Surgery, Massachusetts General Hospital, Boston, USA
3 shared publications Chia Chye Yee
Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, MI
2 shared publications
Latest Publications
- 7 November 2026
- Published by Elsevier in Journal of Surgical Education
- Vol. 83 (12)
Evaluating the Feedback Quality of General Surgery Residents for Open Inguinal Hernia Repair
- 7 November 2026
- Published by Elsevier in Journal of Surgical Education
- Vol. 83 (12)
Evaluating the Feedback Quality of General Surgery Residents for Open Inguinal Hernia Repair
- Open Access
- 15 October 2026
- Published by Springer Nature in Global Surgical Education - Journal of the Association for Surgical Education
- Vol. 5 (1) , 200
Standard setting in the era of workplace-based assessment: a pilot exploration
Purpose: The use of workplace-based assessments (WBAs) in competency-based medical education continues to expand rapidly, and there are emerging methods to use these data to establish a trainee’s competence. However, establishing performance standards based on longitudinally collected operative performance data remains difficult, as existing standard-setting in medical education largely focuses on single tasks, single procedures, or written examinations. The purpose of this pilot study was to (1) assess the feasibility of adapting three established standard-setting methods to WBA ratings scored using the Society for Improving Medical Professional Learning (SIMPL) Operative Performance (OP) score and the convergence of the resulting cut scores in defining a general surgery graduation standard; and (2) identify the methodological and process considerations that could inform future WBA-based standard-setting efforts. Methods: A panel of 10 surgical educators participated in a two-hour virtual Benchmark Panel. Panelists were oriented to the SIMPL OR WBA tool and the OP score (range 372–628). Panelists applied three standard-setting methods to define a cut score corresponding to a “practice-ready” graduation standard for high-value, high-frequency procedures. Each panelist’s mean OP cut score per method was computed and compared using a one-way repeated-measures analysis of variance (ANOVA) with Holm-adjusted pairwise comparisons. The session transcript was analyzed qualitatively using interpretive description methodology to inductively identify themes to inform future iterations. Results: Mean (standard deviation) participant-level OP cut scores were 577 (30) for Angoff, 588 (17) for Bookmark, and 582 (19) for Construct Mapping. There was no association between standard setting method and participant-level OP cut scores [F(2, 18) = 0.63, p = 0.54, generalized η 2 = 0.045]. Holm-adjusted pairwise comparisons were also non-significant (all p > 0.05). Qualitative analysis surfaced themes related to the specific methods, including perceived high utility of Construct Mapping, and overall reflections on the standard-setting process, including the challenge of aligning on the mental model of a “minimally competent graduating trainee” and the difficulty in incorporating variation in case complexity and training experience. A proposed graduation standard of 582, anchored in the Construct Mapping method based on qualitative preference, corresponds to predicted probabilities of earning a practice-ready rating of 88–90% for laparoscopic appendectomy, 83–85% for laparoscopic cholecystectomy, and 76–80% for open partial colectomy. Conclusions: Three established standard-setting methods can be feasibly adapted to longitudinal operative performance WBA data and produced participant-level cut scores that were not statistically different. This convergence across methodologically independent approaches supports the proposed OP-score cut of 582 as an empirically grounded starting point for further multi-stakeholder consensus work. Method selection should be guided by methodological trade-offs in resource requirements, data dependencies, and perceived utility. These findings provide an initial framework for setting defensible, data-informed competency standards in the era of WBA.
- 15 October 2026
- Published by Springer Nature in Global Surgical Education - Journal of the Association for Surgical Education
- Vol. 5 (1) , 200
Open AccessStandard setting in the era of workplace-based assessment: a pilot exploration
Purpose: The use of workplace-based assessments (WBAs) in competency-based medical education continues to expand rapidly, and there are emerging methods to use these data to establish a trainee’s competence. However, establishing performance standards based on longitudinally collected operative performance data remains difficult, as existing standard-setting in medical education largely focuses on single tasks, single procedures, or written examinations. The purpose of this pilot study was to (1) assess the feasibility of adapting three established standard-setting methods to WBA ratings scored using the Society for Improving Medical Professional Learning (SIMPL) Operative Performance (OP) score and the convergence of the resulting cut scores in defining a general surgery graduation standard; and (2) identify the methodological and process considerations that could inform future WBA-based standard-setting efforts. Methods: A panel of 10 surgical educators participated in a two-hour virtual Benchmark Panel. Panelists were oriented to the SIMPL OR WBA tool and the OP score (range 372–628). Panelists applied three standard-setting methods to define a cut score corresponding to a “practice-ready” graduation standard for high-value, high-frequency procedures. Each panelist’s mean OP cut score per method was computed and compared using a one-way repeated-measures analysis of variance (ANOVA) with Holm-adjusted pairwise comparisons. The session transcript was analyzed qualitatively using interpretive description methodology to inductively identify themes to inform future iterations. Results: Mean (standard deviation) participant-level OP cut scores were 577 (30) for Angoff, 588 (17) for Bookmark, and 582 (19) for Construct Mapping. There was no association between standard setting method and participant-level OP cut scores [F(2, 18) = 0.63, p = 0.54, generalized η 2 = 0.045]. Holm-adjusted pairwise comparisons were also non-significant (all p > 0.05). Qualitative analysis surfaced themes related to the specific methods, including perceived high utility of Construct Mapping, and overall reflections on the standard-setting process, including the challenge of aligning on the mental model of a “minimally competent graduating trainee” and the difficulty in incorporating variation in case complexity and training experience. A proposed graduation standard of 582, anchored in the Construct Mapping method based on qualitative preference, corresponds to predicted probabilities of earning a practice-ready rating of 88–90% for laparoscopic appendectomy, 83–85% for laparoscopic cholecystectomy, and 76–80% for open partial colectomy. Conclusions: Three established standard-setting methods can be feasibly adapted to longitudinal operative performance WBA data and produced participant-level cut scores that were not statistically different. This convergence across methodologically independent approaches supports the proposed OP-score cut of 582 as an empirically grounded starting point for further multi-stakeholder consensus work. Method selection should be guided by methodological trade-offs in resource requirements, data dependencies, and perceived utility. These findings provide an initial framework for setting defensible, data-informed competency standards in the era of WBA.
- Open Access
- 28 June 2026
- Published by Springer Nature in Research Square
Standard Setting in the Era of Workplace-Based Assessment: A Pilot Exploration
Purpose: The use of workplace-based assessments (WBAs) in competency-based medical education (CBME) continues to expand rapidly, and there are emerging methods to use this data to establish a trainee’s competence. However, establishing performance standards based on longitudinally collected operative performance data remains difficult, as existing standard-setting in medical education largely focuses on single tasks, single procedures, or written examinations. The purpose of this pilot study was to assess the feasibility and reliability of adapting three established standard-setting methods—Angoff, Bookmark, and Construct Mapping—to WBA ratings scored using the Society for Improving Medical Professional Learning (SIMPL) Operative Performance (OP) score to define a general surgery graduation standard. Methods: A panel of 10 surgical educators participated in a two-hour virtual Benchmark Panel. Panelists were oriented to the SIMPL OR WBA tool and the OP score (range 372–628). Panelists applied three standard-setting methods to define a cut score corresponding to a “practice-ready” graduation standard for high-value, high-frequency procedures. Each panelist’s mean OP cut score per method was computed and compared using a one-way repeated-measures analysis of variance (ANOVA) with Holm-adjusted pairwise comparisons. The session transcript was analyzed qualitatively using interpretive description methodology to inductively identify themes to inform future iterations. Results Mean (standard deviation) participant-level OP cut scores were 577 (30) for Angoff, 588 (17) for Bookmark, and 582 (19) for Construct Mapping. There was no association between standard setting method and participant-level OP cut scores [F(2, 18) = 0.63, p = 0.54, generalized η² = 0.045]. Holm-adjusted pairwise comparisons were also non-significant (all p > 0.05). Qualitative analysis surfaced themes related to the specific methods, including perceived high utility of Construct Mapping, and overall reflections on the standard-setting process, including the challenge of aligning on the mental model of a “minimally competent graduating trainee” and the difficulty in incorporating variation in case complexity and training experience. A proposed graduation standard of 582, anchored in the Construct Mapping method, corresponds to predicted probabilities of earning a practice-ready rating of 88–90% for laparoscopic appendectomy, 83–85% for laparoscopic cholecystectomy, and 76–80% for open partial colectomy. Conclusions Three established standard-setting methods can be feasibly adapted to longitudinal operative performance WBA data and produced participant-level cut scores that were not statistically different. This convergence across methodologically independent approaches supports the proposed OP-score cut of 582 as an empirically grounded starting point for further multi-stakeholder consensus work. Method selection should be guided by methodological trade-offs in resource requirements, data dependencies, and perceived utility. These findings provide an initial framework for setting defensible, data-informed competency standards in the era of WBA.
- 28 June 2026
- Published by Springer Nature in Research Square
Open AccessStandard Setting in the Era of Workplace-Based Assessment: A Pilot Exploration
Purpose: The use of workplace-based assessments (WBAs) in competency-based medical education (CBME) continues to expand rapidly, and there are emerging methods to use this data to establish a trainee’s competence. However, establishing performance standards based on longitudinally collected operative performance data remains difficult, as existing standard-setting in medical education largely focuses on single tasks, single procedures, or written examinations. The purpose of this pilot study was to assess the feasibility and reliability of adapting three established standard-setting methods—Angoff, Bookmark, and Construct Mapping—to WBA ratings scored using the Society for Improving Medical Professional Learning (SIMPL) Operative Performance (OP) score to define a general surgery graduation standard. Methods: A panel of 10 surgical educators participated in a two-hour virtual Benchmark Panel. Panelists were oriented to the SIMPL OR WBA tool and the OP score (range 372–628). Panelists applied three standard-setting methods to define a cut score corresponding to a “practice-ready” graduation standard for high-value, high-frequency procedures. Each panelist’s mean OP cut score per method was computed and compared using a one-way repeated-measures analysis of variance (ANOVA) with Holm-adjusted pairwise comparisons. The session transcript was analyzed qualitatively using interpretive description methodology to inductively identify themes to inform future iterations. Results Mean (standard deviation) participant-level OP cut scores were 577 (30) for Angoff, 588 (17) for Bookmark, and 582 (19) for Construct Mapping. There was no association between standard setting method and participant-level OP cut scores [F(2, 18) = 0.63, p = 0.54, generalized η² = 0.045]. Holm-adjusted pairwise comparisons were also non-significant (all p > 0.05). Qualitative analysis surfaced themes related to the specific methods, including perceived high utility of Construct Mapping, and overall reflections on the standard-setting process, including the challenge of aligning on the mental model of a “minimally competent graduating trainee” and the difficulty in incorporating variation in case complexity and training experience. A proposed graduation standard of 582, anchored in the Construct Mapping method, corresponds to predicted probabilities of earning a practice-ready rating of 88–90% for laparoscopic appendectomy, 83–85% for laparoscopic cholecystectomy, and 76–80% for open partial colectomy. Conclusions Three established standard-setting methods can be feasibly adapted to longitudinal operative performance WBA data and produced participant-level cut scores that were not statistically different. This convergence across methodologically independent approaches supports the proposed OP-score cut of 582 as an empirically grounded starting point for further multi-stakeholder consensus work. Method selection should be guided by methodological trade-offs in resource requirements, data dependencies, and perceived utility. These findings provide an initial framework for setting defensible, data-informed competency standards in the era of WBA.
- 1 June 2026
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 18 (3) , 334-350
Graduate Medical Education Data in the United States: Availability and Accessibility Across National Organizations
Background: Longitudinal, multi-organizational data have enormous potential for improving graduate medical education (GME) yet aggregating and sharing that data remain difficult. Objective: To determine what data is collected and how data is shared by US medical education organizations, as a next step toward expanded sharing of GME data. Methods: Ten multispecialty US medical education and licensing organizations that participated in the 2019 National Academies of Sciences, Engineering, and Medicine conference completed a pre-meeting data survey. Leaders from 9 of these organizations participated in semi-structured interviews conducted between January 2021 and May 2022. Content analysis of survey data and interview transcripts were used to construct a data inventory. Each of the 9 interviewed organizations reviewed and updated their inventory in January and February 2024. Inductive analyses of interview transcripts were conducted to identify additional themes. Results: The pre-meeting survey and interviews had a 100% (10 total) and 90% (9 of 10) response rate, respectively. Study participants collect a large amount of GME-related data, some of which is duplicative. None collect longer-term outcomes, including patient outcomes. There is also no common physician identifier used by all organizations, although Association of American Medical Colleges ID and National Provider Identifier have been adopted most widely. All organizations share data to varying degrees. Two themes emerged regarding ongoing efforts to expand data sharing: perceived benefits of a harmonized system and challenges to implementation. Conclusions: Data are increasingly available from organizations collecting national GME data in the United States.
- 1 June 2026
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 18 (3) , 334-350
Graduate Medical Education Data in the United States: Availability and Accessibility Across National Organizations
Background: Longitudinal, multi-organizational data have enormous potential for improving graduate medical education (GME) yet aggregating and sharing that data remain difficult. Objective: To determine what data is collected and how data is shared by US medical education organizations, as a next step toward expanded sharing of GME data. Methods: Ten multispecialty US medical education and licensing organizations that participated in the 2019 National Academies of Sciences, Engineering, and Medicine conference completed a pre-meeting data survey. Leaders from 9 of these organizations participated in semi-structured interviews conducted between January 2021 and May 2022. Content analysis of survey data and interview transcripts were used to construct a data inventory. Each of the 9 interviewed organizations reviewed and updated their inventory in January and February 2024. Inductive analyses of interview transcripts were conducted to identify additional themes. Results: The pre-meeting survey and interviews had a 100% (10 total) and 90% (9 of 10) response rate, respectively. Study participants collect a large amount of GME-related data, some of which is duplicative. None collect longer-term outcomes, including patient outcomes. There is also no common physician identifier used by all organizations, although Association of American Medical Colleges ID and National Provider Identifier have been adopted most widely. All organizations share data to varying degrees. Two themes emerged regarding ongoing efforts to expand data sharing: perceived benefits of a harmonized system and challenges to implementation. Conclusions: Data are increasingly available from organizations collecting national GME data in the United States.
- 1 April 2026
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 18 (2) , 216-224
An Exploration of Competency Assessment in Graduate Medical Education
- 1 April 2026
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 18 (2) , 216-224
An Exploration of Competency Assessment in Graduate Medical Education
- 23 September 2025
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (12) , 103705
“Stepping Out of a Comfort Zone”: Evaluating the Impact of a Leadership Skills Curriculum on Surgical Residents’ Transition to Senior Roles
- 23 September 2025
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (12) , 103705
“Stepping Out of a Comfort Zone”: Evaluating the Impact of a Leadership Skills Curriculum on Surgical Residents’ Transition to Senior Roles
- 22 December 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (2) , 103375
Characterizing the Relationships Amongst Psychological Safety, the Learning Environment, and Well-Being in Surgical Faculty and Trainees
- 22 December 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (2) , 103375
Characterizing the Relationships Amongst Psychological Safety, the Learning Environment, and Well-Being in Surgical Faculty and Trainees
- 9 November 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (1) , 103318
Readiness of Graduating General Surgery Residents To Perform Common Pediatric Surgery Procedures
- 9 November 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (1) , 103318
Readiness of Graduating General Surgery Residents To Perform Common Pediatric Surgery Procedures
- 30 October 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (1) , 103297-10
Operative Performance and Autonomy Across Training Years: Does a Preliminary Year Matter?
- 30 October 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 82 (1) , 103297-10
Operative Performance and Autonomy Across Training Years: Does a Preliminary Year Matter?
- 29 March 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 81 (5) , 626-638
National Profile of the ACGME Milestones 1.0 and 2.0 within General Surgery: A Seven-Year National Study from 2014 to 2021
- 29 March 2024
- Published by Elsevier in Journal of Surgical Education
- Vol. 81 (5) , 626-638
National Profile of the ACGME Milestones 1.0 and 2.0 within General Surgery: A Seven-Year National Study from 2014 to 2021