John Norcini

Reflecting on the long‐observed dichotomy between ‘objective’ and ‘subjective’ assessments, Norcini suggests that all assessments are based on judgments and, hence, we need to be cautious when drawing implications from the distinction.

Introduction: Changing health care requires changing medical education. In this position paper it is suggested that subsequent innovations in medical education each had their specific strengths and shortcomings. What they have, however, in common is that they place the medical student and their competencies at their center. Innovation in medical education is inward looking. Discussion: The authors propose a perspective on the medical curriculum in which the patient, their family, and the surrounding community take center stage. They argue that present medical education cannot adequately respond to the great challenges to population health: an aging population, the obesity epidemic, and future pandemics of new diseases due to population growth, urbanization, and antimicrobial resistance, particularly because these challenges cannot be dealt with by the medical sciences alone but need deep understanding of the social sciences as well. In addition, the practice of health care is changing: effective health care demands a close partnership between the health care system and the medical school which is mostly lacking, cooperation with other health professions is becoming more and more necessary in response to the increasing complexity of health care, patients and their families are required to play a more active role in their health, medical error threatening patient safety is becoming to be seen as a huge problem, and the emergence of artificial intelligence in education and practice, all requiring transformation of medical education. Conclusion: The present contribution suggests eight such transformations necessary to create a truly patient- and population-centered medical curriculum.

Submitted: 4 November 2023 Accepted: 20 November 2023 Published online: 2 April, TAPS 2024, 9(2),98-100 https://doi.org/10.29060/TAPS.2024-9-2/PV3165 John Norcini Department of Psychiatry, SUNY Upstate Medical University, United States of America I. INTRODUCTION Over the past 25 years, the Asia Pacific region has seen striking growth in the scholarship of health professions education, and it is poised […]

Purpose The United States Medical Licensing Examination (USMLE) comprises a series of assessments required for the licensure of United States MD-trained graduates as well as those who are trained internationally. Demonstration of a relationship between these examinations and outcomes of care is desirable for a process seeking to provide patients with safe and effective health care. Method This was a retrospective cohort study of 196,881 hospitalizations in Pennsylvania over a 3-year period (January 1, 2017–December 31, 2019), for 5 primary diagnoses: heart failure, acute myocardial infarction, stroke, pneumonia, or chronic obstructive pulmonary disease. The 1,765 attending physicians for these hospitalizations self-identified as family physicians or general internists. A converted score based on USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores was available and the outcome measures were in-hospital mortality and log length of stay (LOS). The research team controlled for characteristics of patients, hospitals, and physicians. Results For in-hospital mortality, the adjusted odds ratio was 0.94 (95% confidence interval [CI] 0.90, .99; P < .02). Each standard deviation increase in the converted score was associated with a 5.51% reduction in the odds of in-hospital mortality. For log LOS, the adjusted estimate was 0.99 (95% CI = 0.98, 0.99; P < .001). Each standard deviation increase in the converted score was associated with a 1.34% reduction in log LOS. Conclusions Better provider USMLE performance was associated with lower in-hospital mortality and shorter log LOS for patients, although the magnitude of the latter is unlikely to be of practical significance. These findings add to the body of evidence that examines the validity of the USMLE licensure program.

In this issue, Ryan and colleagues underscore the need for criterion-based assessments in the context of competency-based curricula in undergraduate medical education (UME). They also point out that the same scores are often interpreted from a norm-referenced perspective to support the admissions process for residency training. This problem is not unique to UME because in graduate medical education (GME), the same assessments are often used for both decision making and providing feedback. Unfortunately, an assessment with 2 purposes is neither optimal nor efficient for either purpose and may be accompanied by significant side effects. One approach to addressing these challenges is to develop a system of assessment that addresses both purposes but where each component is focused on a single purpose. This leads to alignment and transparency from purpose to test content and from test content to score interpretation and/or feedback. It ensures that the test material is optimized for the task, that individual assessments are constructed to enhance the validity of their scores, and that undesirable side effects are limited.

Structured Self-Regulated Learning (SSRL) diaries have the potential for combining the development and assessment of a student’s SRL processes over time. The aim of this study was to evaluate the extent to which an SSRL diary can develop SRL and provide a reliable longitudinal assessment of SRL development in academically low-achieving undergraduate medical students. We conducted a quasi-experimental study with low-achieving medical students at Tehran University of Medical Sciences. The intervention was a weekly SSRL diary, with 21 items in two parts (before and after studying) that was integrated with weekly explicit SRL training. A repeated measures ANOVA was performed to assess the participants' SRL development across time. We conducted generalizability theory analysis in two designs; the first was to assess the reliability of the total diary score in assessing low-achieving medical students’ SRL and the second was to assess the efficacy of the four-week intervention results in improving the low-achieving medical students’ SRL. Each participant (n = 20) completed four SSRL diaries. There were significant positive changes during the intervention in the students’ measures of total SRL, time estimation of study, time spent on study, concentration, elaboration, organization, self-monitoring, and self-evaluation. The absolute and relative generalizability coefficients for the first design, which indicates the reliability of the students’ SRL scores, were 0.71 and 0.77. The absolute and relative generalizability coefficients for the second design, which presents the reliability of the improvement of students’ SRL across time were 0.79 and 0.87. The results of this study suggest that SSRL diaries combined with explicit training can provide an effective and reliable method for combining the development and assessment of SRL in low-achieving medical students’ SRL over time.

Objective: The aim of this paper is to provide an overview of the major quality assurance strategies, accreditation and licensure, in health professions education. It explores the nature of these regulatory processes using Brazil and the United States as examples because these large systems are at different ends of the developmental continuum. For each, it describes the tensions that arise, offers a critical synthesis of the evidence and maps out future directions. Results: Given wide variability among operating medical schools in curricular design, length of study, resources and facilities for clinical training and supervision, the nature of regulatory bodies varies considerably. Nonetheless, they share tensions related purpose and process including quality assurance versus quality improvement, outcomes versus process and continuous versus episodic evaluations and assessments. Clear evidence of effectiveness, especially for accreditation, is scarce and difficult to obtain, particularly as it relates to health outcomes. Conclusions: Regulatory processes need to be built around clear definitions of the goals for each stage of professional development, the current movement towards competency‐based education and the variable durations of medical education. These changes must motivate revisions in the content and process of programmes for accreditation and licensure, complimentary efforts towards quality of care, and stimulate a significant research effort.

Objective To determine whether internists’ initial specialty certification and the maintenance of that certification (MOC) is associated with lower in-hospital mortality for their patients with acute myocardial infarction (AMI) or congestive heart failure (CHF). Design Retrospective cohort study of hospitalisations in Pennsylvania, USA, from 2012 to 2017. Setting All hospitals in Pennsylvania. Participants All 184 115 hospitalisations for primary diagnoses of AMI or CHF where the attending physician was a self-designated internist. Primary outcome measure In-hospital mortality. Results Of the 2575 physicians, 2238 had initial certification and 820 were eligible for MOC. After controlling for patient demographics and clinical characteristics, hospital-level factors and physicians’ demographic and medical school characteristics, both initial certification and MOC were associated with lower mortality. The adjusted OR for initial certification was 0.835 (95% CI 0.756 to 0.922; pConclusions Initial certification was associated with lower mortality for AMI or CHF. Moreover, for patients whose physicians had initial certification, an additional advantage was associated with its maintenance.

Article Improving the physical exam: a new assessment and evaluation tool for physical examination skills was published on March 17, 2022 in the journal Diagnosis (volume 0, issue 0).