Janet Riddle
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Janet Riddle has a total of 74 co-authors. Here are the top co-authors by number of shared publications:
- Georges Bordage
University of Illinois at Chicago
5 shared publications - Rachel Yudkowsky
Department of Medical Education, University of Illinois College of Medicine, Chicago, Illinois
4 shared publications - Yoon Soo Park
Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois
3 shared publications - Anna Therese Cianciolo
Department of Medical Education, Southern Illinois University School of Medicine, Springfield, USA
2 shared publications Deborah Simpson
Medical Education Program Director, Advocate Aurora Health, Adjunct Clinical Professor of Family Medicine, University of Wisconsin School of Medicine and Public Health and Medical College of Wisconsin
2 shared publicationsTali Lowenstein
Department of Medical Education, College of Medicine, University of Illinois at Chicago, Chicago, Illinois, USA
2 shared publications
Latest Publications
- Open Access
- 2 August 2021
- Published by Association of American Medical Colleges in MedEdPORTAL
- Vol. 17
Fundamentals of Scholarly Peer Review: A Workshop for Health Professions Educators on Practicing Scholarly Citizenship
Introduction Education scholarship requires peer reviewers. For novice scholars, reviewing is an important developmental activity that cultivates deeper participation in the scholarship community. ...
- 2 August 2021
- Published by Association of American Medical Colleges in MedEdPORTAL
- Vol. 17
Open AccessFundamentals of Scholarly Peer Review: A Workshop for Health Professions Educators on Practicing Scholarly Citizenship
Introduction Education scholarship requires peer reviewers. For novice scholars, reviewing is an important developmental activity that cultivates deeper participation in the scholarship community. ...
- Open Access
- 1 December 2020
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 12 (6) , 783-784
Blueprinting Evaluation Evidence: Data Sources and Methods
- 1 December 2020
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 12 (6) , 783-784
Open AccessBlueprinting Evaluation Evidence: Data Sources and Methods
- 1 August 2020
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 12 (4) , 499-500
Getting the Evaluation Focus Clear: A Shared Understanding of What Is Being Evaluated
Article Citation: Janet M. Riddle, Amy Halverson, and Michelle Barnes (2020) Getting the Evaluation Focus Clear: A Shared Understanding of What Is Being Evaluated. Journal of Graduate Medical Education: August 2020, Vol. 12, No. 4, pp. 499-500. https://doi.org/10.4300/JGME-D-20-00701.1
- 1 August 2020
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 12 (4) , 499-500
Getting the Evaluation Focus Clear: A Shared Understanding of What Is Being Evaluated
Article Citation: Janet M. Riddle, Amy Halverson, and Michelle Barnes (2020) Getting the Evaluation Focus Clear: A Shared Understanding of What Is Being Evaluated. Journal of Graduate Medical Education: August 2020, Vol. 12, No. 4, pp. 499-500. https://doi.org/10.4300/JGME-D-20-00701.1
- 1 January 2019
- Published by Elsevier in Journal of Pain and Symptom Management
- Vol. 57 (1) , 108-111
Pilot Qualitative Study of Informal Teachers in Interprofessional Collaboration and Practice
- 1 January 2019
- Published by Elsevier in Journal of Pain and Symptom Management
- Vol. 57 (1) , 108-111
Pilot Qualitative Study of Informal Teachers in Interprofessional Collaboration and Practice
- Open Access
- 11 May 2018
- Published by Escola Bahiana de Medicina e Saude Publica in International Journal of Education and Health
- Vol. 2 (1) , 6-8
Lessons from practice
- 11 May 2018
- Published by Escola Bahiana de Medicina e Saude Publica in International Journal of Education and Health
- Vol. 2 (1) , 6-8
Open AccessLessons from practice
- 2 April 2018
- Published by Taylor & Francis in Teaching and Learning in Medicine
- Vol. 31 (1) , 7-16
Conversation Starter: Advancing the Theory of Peer-Assisted Learning
This Conversation Starter article uses four selected abstracts, one each from the four regional Association of American Medical Colleges (AAMC) Group on Educational Affairs (CGEA) 2018 spring meetings, as a springboard for unpacking the definition of peer-assisted learning (PAL). The aim of this article is to prompt deeper reflection on this phenomenon and, in so doing, to foster scholarly program evaluation of this widely adopted instructional approach. This analysis calls for a more nuanced definition of PAL, one that emphasizes process over structure, one that stimulates examination of “doing” PAL and how this affects the personal and professional development of all involved.
- 2 April 2018
- Published by Taylor & Francis in Teaching and Learning in Medicine
- Vol. 31 (1) , 7-16
Conversation Starter: Advancing the Theory of Peer-Assisted Learning
This Conversation Starter article uses four selected abstracts, one each from the four regional Association of American Medical Colleges (AAMC) Group on Educational Affairs (CGEA) 2018 spring meetings, as a springboard for unpacking the definition of peer-assisted learning (PAL). The aim of this article is to prompt deeper reflection on this phenomenon and, in so doing, to foster scholarly program evaluation of this widely adopted instructional approach. This analysis calls for a more nuanced definition of PAL, one that emphasizes process over structure, one that stimulates examination of “doing” PAL and how this affects the personal and professional development of all involved.
- 23 September 2015
- Published by Taylor & Francis in Medical Teacher
- Vol. 38 (2) , 141-149
Twelve tips for developing, implementing, and sustaining medical education fellowship programs: Building on new trends and solid foundations
Medical education fellowship programs (MEFPs) are a form of faculty development contributing to an organization’s educational mission and participants’ career development. Building an MEFP requires a systematic design, implementation, and evaluation approach which aligns institutional and individual faculty goals. Implementing an MEFP requires a team of committed individuals who provide expertise, guidance, and mentoring. Qualified MEFP directors should utilize instructional methods that promote individual and institutional short and long term growth. Directors must balance the use of traditional design, implementation, and evaluation methodologies with advancing trends that may support or threaten the acceptability and sustainability of the program. Drawing on the expertise of 28 MEFP directors, we provide twelve tips as a guide to those implementing, sustaining, and/or growing a successful MEFP whose value is demonstrated by its impacts on participants, learners, patients, teaching faculty, institutions, the greater medical education community, and the population’s health.
- 23 September 2015
- Published by Taylor & Francis in Medical Teacher
- Vol. 38 (2) , 141-149
Twelve tips for developing, implementing, and sustaining medical education fellowship programs: Building on new trends and solid foundations
Medical education fellowship programs (MEFPs) are a form of faculty development contributing to an organization’s educational mission and participants’ career development. Building an MEFP requires a systematic design, implementation, and evaluation approach which aligns institutional and individual faculty goals. Implementing an MEFP requires a team of committed individuals who provide expertise, guidance, and mentoring. Qualified MEFP directors should utilize instructional methods that promote individual and institutional short and long term growth. Directors must balance the use of traditional design, implementation, and evaluation methodologies with advancing trends that may support or threaten the acceptability and sustainability of the program. Drawing on the expertise of 28 MEFP directors, we provide twelve tips as a guide to those implementing, sustaining, and/or growing a successful MEFP whose value is demonstrated by its impacts on participants, learners, patients, teaching faculty, institutions, the greater medical education community, and the population’s health.
- 1 July 2014
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 89 (7) , 1057-1062
Clinically Discriminating Checklists Versus Thoroughness Checklists
Purpose High-quality checklists are essential to performance test score validity. Prior research found that physical exam checklists of items that clinically discriminated between competing diagnoses provided more generalizable scores than all-encompassing thoroughness checklists. The purpose of this study was to compare validity evidence for clinically discriminating versus thoroughness checklists, hypothesizing that evidence would favor the former. Method Faculty at four Chicago-area medical schools developed six standardized patient (SP) cases with checklists of about 20 items (“thoroughness [long] checklists”). Four clinicians identified a subset of items that clinically discriminated between competing diagnoses of each case (“clinically discriminating [short] checklists”). Cases were administered to 155 University of Illinois at Chicago fourth-year medical students during their 2011 Clinical Skills Examination (CSE). Validity evidence was compared for CSE scores based on thoroughness versus clinically discriminating checklist items. Results Validity evidence favoring clinically discriminating checklists included response process: greater SP checklist accuracy (kappa = 0.75 for long and 0.84 for short checklists, P < .05); internal structure: better item discrimination (0.28 long, 0.42 short, P < .001); internal consistency reliability (0.80 long, 0.92 short); standard error of measurement (z score 8.87 long, 8.05 short); and generalizability (G = 0.504 long, 0.533 short). There were no significant differences overall in relevance ratings, item difficulty, or cut scores of long versus short checklist items. Conclusions Limiting checklist items to those affecting diagnostic decisions resulted in better accuracy and psychometric indices. Thoroughness items performed without thinking do not reflect clinical reasoning ability and contribute construct-irrelevant variance to scores.
- 1 July 2014
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 89 (7) , 1057-1062
Clinically Discriminating Checklists Versus Thoroughness Checklists
Purpose High-quality checklists are essential to performance test score validity. Prior research found that physical exam checklists of items that clinically discriminated between competing diagnoses provided more generalizable scores than all-encompassing thoroughness checklists. The purpose of this study was to compare validity evidence for clinically discriminating versus thoroughness checklists, hypothesizing that evidence would favor the former. Method Faculty at four Chicago-area medical schools developed six standardized patient (SP) cases with checklists of about 20 items (“thoroughness [long] checklists”). Four clinicians identified a subset of items that clinically discriminated between competing diagnoses of each case (“clinically discriminating [short] checklists”). Cases were administered to 155 University of Illinois at Chicago fourth-year medical students during their 2011 Clinical Skills Examination (CSE). Validity evidence was compared for CSE scores based on thoroughness versus clinically discriminating checklist items. Results Validity evidence favoring clinically discriminating checklists included response process: greater SP checklist accuracy (kappa = 0.75 for long and 0.84 for short checklists, P < .05); internal structure: better item discrimination (0.28 long, 0.42 short, P < .001); internal consistency reliability (0.80 long, 0.92 short); standard error of measurement (z score 8.87 long, 8.05 short); and generalizability (G = 0.504 long, 0.533 short). There were no significant differences overall in relevance ratings, item difficulty, or cut scores of long versus short checklist items. Conclusions Limiting checklist items to those affecting diagnostic decisions resulted in better accuracy and psychometric indices. Thoroughness items performed without thinking do not reflect clinical reasoning ability and contribute construct-irrelevant variance to scores.
- 9 June 2014
- Published by Springer Nature
- p. 77-91
Teaching Clinical Skills
- 8 May 2014
- Published by Wiley in Medical Education
- Vol. 48 (6) , 614-622
Validity evidence of resident competency ratings and the identification of problem residents
Objectives: This study examined validity evidence of end‐of‐rotation evaluations used to measure progress toward mastery of core competencies in residents. In addition, this study investigated whether end‐of‐rotation evaluations can be used to detect problem residents during their training. Methods: Historical data for a 4‐year period (2009–2012), containing 4986 observations of 291 internal medicine residents, were examined. Residents were observed and assessed by fellows, faculty members and programme directors on nine domains, including the six Accreditation Council for Graduate Medical Education core competencies, as part of their end‐of‐rotation evaluations. Descriptive statistics were used to collect evidence of the response process. Correlations between competencies and a generalisability study were used to examine the internal structure of the end‐of‐rotation evaluations. Hierarchical regression was used to estimate the increase in scores across years of training. Scores on end‐of‐rotation evaluations were compared with trainees identified as problem residents by programme directors. Results: Compared with fellows, faculty and programme directors had significantly greater variability in assigning scores across different competencies. Correlations between competencies ranged from 0.69 to 0.92. The reliability of end‐of‐rotation evaluations was adequate (fellows, phi coefficient [φ] = 0.68; faculty [including programme directors], φ = 0.71). Mean scores increased by 0.21 points (95% confidence interval 0.18–0.24) per postgraduate year. Mean scores were significantly correlated with classification as a problem resident ( r = 0.33, p < 0.001); problem residents also had significantly lower ratings across all competencies during PGY‐1 compared with all other residents. Conclusions: End‐of‐rotation evaluations are a useful method of measuring the growth in resident performance associated with core competencies when sufficient numbers of end‐of‐rotation evaluation scores are used. Furthermore, end‐of‐rotation evaluation scores provide preliminary evidence with which to detect and predict problem residents in subsequent postgraduate training years.
- 8 May 2014
- Published by Wiley in Medical Education
- Vol. 48 (6) , 614-622
Validity evidence of resident competency ratings and the identification of problem residents
Objectives: This study examined validity evidence of end‐of‐rotation evaluations used to measure progress toward mastery of core competencies in residents. In addition, this study investigated whether end‐of‐rotation evaluations can be used to detect problem residents during their training. Methods: Historical data for a 4‐year period (2009–2012), containing 4986 observations of 291 internal medicine residents, were examined. Residents were observed and assessed by fellows, faculty members and programme directors on nine domains, including the six Accreditation Council for Graduate Medical Education core competencies, as part of their end‐of‐rotation evaluations. Descriptive statistics were used to collect evidence of the response process. Correlations between competencies and a generalisability study were used to examine the internal structure of the end‐of‐rotation evaluations. Hierarchical regression was used to estimate the increase in scores across years of training. Scores on end‐of‐rotation evaluations were compared with trainees identified as problem residents by programme directors. Results: Compared with fellows, faculty and programme directors had significantly greater variability in assigning scores across different competencies. Correlations between competencies ranged from 0.69 to 0.92. The reliability of end‐of‐rotation evaluations was adequate (fellows, phi coefficient [φ] = 0.68; faculty [including programme directors], φ = 0.71). Mean scores increased by 0.21 points (95% confidence interval 0.18–0.24) per postgraduate year. Mean scores were significantly correlated with classification as a problem resident ( r = 0.33, p < 0.001); problem residents also had significantly lower ratings across all competencies during PGY‐1 compared with all other residents. Conclusions: End‐of‐rotation evaluations are a useful method of measuring the growth in resident performance associated with core competencies when sufficient numbers of end‐of‐rotation evaluation scores are used. Furthermore, end‐of‐rotation evaluation scores provide preliminary evidence with which to detect and predict problem residents in subsequent postgraduate training years.