AKAndrew E. Krumm

Andrew E. Krumm

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Andrew E. Krumm has a total of 156 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Evaluating the Feedback Quality of General Surgery Residents for Open Inguinal Hernia Repair

    by Rebecca Moreci, Kayla M. Marcotte, Alyssa Pradarelli, Gurjit Sandhu, Julie Evans, Andrew E. Krumm, Brian C. George, Stefanie S. Sebok-Syer

  • Open Access

    Standard setting in the era of workplace-based assessment: a pilot exploration

    by Alyssa A. Pradarelli, Kayla M. Marcotte, Brian C. George, Tyler J. Loftus, James R. Korndorffer, David T. Hughes, Gifty Kwakye, Sophia K. McKinley, Erin M. White, James N. Lau, et al.

    Purpose: The use of workplace-based assessments (WBAs) in competency-based medical education continues to expand rapidly, and there are emerging methods to use these data to establish a trainee’s competence. However, establishing performance standards based on longitudinally collected operative performance data remains difficult, as existing standard-setting in medical education largely focuses on single tasks, single procedures, or written examinations. The purpose of this pilot study was to (1) assess the feasibility of adapting three established standard-setting methods to WBA ratings scored using the Society for Improving Medical Professional Learning (SIMPL) Operative Performance (OP) score and the convergence of the resulting cut scores in defining a general surgery graduation standard; and (2) identify the methodological and process considerations that could inform future WBA-based standard-setting efforts. Methods: A panel of 10 surgical educators participated in a two-hour virtual Benchmark Panel. Panelists were oriented to the SIMPL OR WBA tool and the OP score (range 372–628). Panelists applied three standard-setting methods to define a cut score corresponding to a “practice-ready” graduation standard for high-value, high-frequency procedures. Each panelist’s mean OP cut score per method was computed and compared using a one-way repeated-measures analysis of variance (ANOVA) with Holm-adjusted pairwise comparisons. The session transcript was analyzed qualitatively using interpretive description methodology to inductively identify themes to inform future iterations. Results: Mean (standard deviation) participant-level OP cut scores were 577 (30) for Angoff, 588 (17) for Bookmark, and 582 (19) for Construct Mapping. There was no association between standard setting method and participant-level OP cut scores [F(2, 18) = 0.63, p = 0.54, generalized η 2 = 0.045]. Holm-adjusted pairwise comparisons were also non-significant (all p > 0.05). Qualitative analysis surfaced themes related to the specific methods, including perceived high utility of Construct Mapping, and overall reflections on the standard-setting process, including the challenge of aligning on the mental model of a “minimally competent graduating trainee” and the difficulty in incorporating variation in case complexity and training experience. A proposed graduation standard of 582, anchored in the Construct Mapping method based on qualitative preference, corresponds to predicted probabilities of earning a practice-ready rating of 88–90% for laparoscopic appendectomy, 83–85% for laparoscopic cholecystectomy, and 76–80% for open partial colectomy. Conclusions: Three established standard-setting methods can be feasibly adapted to longitudinal operative performance WBA data and produced participant-level cut scores that were not statistically different. This convergence across methodologically independent approaches supports the proposed OP-score cut of 582 as an empirically grounded starting point for further multi-stakeholder consensus work. Method selection should be guided by methodological trade-offs in resource requirements, data dependencies, and perceived utility. These findings provide an initial framework for setting defensible, data-informed competency standards in the era of WBA.

  • Graduate Medical Education Data in the United States: Availability and Accessibility Across National Organizations

    by Angela E. Thelen, Rebecca S. Gates, Deena Khamees, Mary R. C. Haas, Alyssa A. Pradarelli, John Burkhardt, Andrew E. Krumm, Debra Weinstein, Brian C. George

    Background: Longitudinal, multi-organizational data have enormous potential for improving graduate medical education (GME) yet aggregating and sharing that data remain difficult. Objective: To determine what data is collected and how data is shared by US medical education organizations, as a next step toward expanded sharing of GME data. Methods: Ten multispecialty US medical education and licensing organizations that participated in the 2019 National Academies of Sciences, Engineering, and Medicine conference completed a pre-meeting data survey. Leaders from 9 of these organizations participated in semi-structured interviews conducted between January 2021 and May 2022. Content analysis of survey data and interview transcripts were used to construct a data inventory. Each of the 9 interviewed organizations reviewed and updated their inventory in January and February 2024. Inductive analyses of interview transcripts were conducted to identify additional themes. Results: The pre-meeting survey and interviews had a 100% (10 total) and 90% (9 of 10) response rate, respectively. Study participants collect a large amount of GME-related data, some of which is duplicative. None collect longer-term outcomes, including patient outcomes. There is also no common physician identifier used by all organizations, although Association of American Medical Colleges ID and National Provider Identifier have been adopted most widely. All organizations share data to varying degrees. Two themes emerged regarding ongoing efforts to expand data sharing: perceived benefits of a harmonized system and challenges to implementation. Conclusions: Data are increasingly available from organizations collecting national GME data in the United States.

  • An Exploration of Competency Assessment in Graduate Medical Education

    by Kayla M. Marcotte, Alyssa Pradarelli, Brian C. George, Jamila K. Picart, Kenneth L. Abbott, Philip Hong, Tyler Loftus, Joseph B. Zwischenberger, Andrew E. Krumm


  • EQIP 2023-2025: Increased Usability and Sustainability

    by Amit R.T. Joshi, Andrew Krumm, Daniel M. Relles, Kristen Conrad-Schnetz, Kyla Terhune, Brian George, Jennifer Choi, David T. Harrington, Stephen M. Kavic, Elango Edhayan, et al.

  • What is lost in an average? Identifying distinct post‐radical prostatectomy functional recovery profiles

    by Patrick Lewicki, Kevin Ginsburg, Nnenaya Mmonu, Corinne Labardee, Anna Johnson, James Peabody, Adam Gadzinski, Alice Semerjian, Tudor Borza, Brian R. Lane, et al.

    Objective: To describe, via latent variable mixture modelling, distinct post‐radical prostatectomy (RP) patient‐reported outcome (PRO) recovery profiles, which are positioned to complement currently disseminated statistical averages for shared decision‐making. Patients and Methods: Patients undergoing RP and completing the 26‐item Expanded Prostate Cancer Index Composite 12 months after surgery were identified from the Michigan Urological Surgery Improvement Collaborative data registry. Hierarchical cluster analysis and latent variable mixture modelling was applied to urinary incontinence (UI) and sexual function (SF) recovery scores, and final models chosen based on optimal performance. Results: A total of 3956 patients comprised the study cohort. Three distinct UI profiles were identified with prevalence of 49%, 37% and 14% from best to worst recovery, respectively. Four distinct SF profiles were identified with prevalence of 14%, 24%, 42%, and 20%, from best to worst recovery, respectively. The last two SF profiles had similar function scores but differed based on perception of function being bothersome. Limitations include incomplete PRO capture, which may introduce bias. Conclusions: We identify distinct UI and SF recovery profiles and their prevalence from a large, prospectively maintained registry, potentially improving interpretability of PRO data for decision making.


  • Ethical considerations of using learning analytics in medical education: a critical review

    by Kayla Marcotte, Phillip Yang, M. Andrew Millis, Christian J. Vercler, Stefanie S. Sebok-Syer, Andrew E. Krumm, Brian C. George

    Learning analytics are increasingly used in medical education to analyze data and make decisions about learners’ abilities. While there are many potential benefits of using learning analytics to drive improvement in medical education, there are also ethical concerns surrounding how this may affect learners and their patients. We conducted a critical review of studies that use learning analytics and big data within medical education. Using guidelines established by Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), relevant articles were identified in MEDLINE (PubMed) and SocINDEX databases from inception to April 2021. Detailed data abstraction was performed across studies to identify current uses of learning analytics and identify potential ethical concerns. Eighteen articles met the search criteria. Our analysis identified the use of learning analytics and big data in four aspects of medical education: (1) the learning process and pedagogy; (2) retrospective assessment; (3) prospective assessment; and (4) improvement of education. We identified some ethical concerns surrounding the use of learning analytics and big data, including the (1) trustworthiness of data; (2) reliability of methodology; (3) privacy, confidentiality, and management of data; and (4) labeling of learners as “problematic.” Using Beauchamp and Childress’s biomedical ethics as a framework, we identified potential consequences of using learning analytics for learners within the principles of beneficence, nonmaleficence, autonomy, and justice. As learning analytics becomes more widespread in medical education, examining and mitigating potential harm towards learners is imperative.


  • Open Access

    Proximity to Practice: The Role of Technology in the Next Era of Assessment

    by Andrew E. Krumm, Hollis Lai, Kayla Marcotte, Tavinder K. Ark, Victoria Yaneva, Saad Chahine

    Perspectives on Medical Education’s mission is to support and enrich collaborative scholarship between education researchers and clinical educators, and to advance new knowledge regarding clinical education practices. The journal is the official journal of the The Netherlands Association of Medical Education (NVMO), which sponsors the journal as an Open Access journal that is free to read immediately upon publication. To publish in the journal, authors are required to pay an author processing charge of £1026 upon publication. The journal is indexed in key databases, including MEDLINE, Web of Science, Google Scholar, and Scopus and has received the DOAJ seal. In 2023, the journal was awarded an impact factor of 4.8. The journal is overseen by the Editor-in-Chief Professor Erik Driessen, PhD. He is provided by guidance by an international editorial board of leading scholars with members representing the different parental disciplines in the field of medical education, e.g. clinicians, social scientists, biomedical scientists, statisticians, and linguists.


  • Open Access

    Digital Evidence: Revisiting Assumptions at the Intersection of Technology and Assessment

    by Andrew E. Krumm, Saad Chahine, Abigail M. Schuh, Daniel J. Schumacher, Sondra Zabar, Brian C. George, Kayla Marcotte, Stefanie S. Sebok-Syer, Michael A. Barone, Alina Smirnova

    The increasing use of technology in health care and health professions education is an invitation to examine how digital sources of evidence are used in making assessment claims. In this paper, we describe how four sets of terms—primary and secondary data; structured and unstructured data; development and use; and deterministic and generative—can aid in examining how data from digital sources are used in evaluating what learners know and can do. Drawing on multiple examples, this paper shows how the four sets of terms can help both developers and users of technology-based assessment systems.