Kulamakan Kulasegaram
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Kulamakan Kulasegaram has a total of 130 co-authors. Here are the top co-authors by number of shared publications:
- Ryan Brydges
Department of Medicine, University of Toronto
8 shared publications - Adam G. Gavarkovs
Division of Continuing Professional Development, Faculty of Medicine, University of British Columbia
5 shared publications - Erin Miller
School of Physical Therapy, Faculty of Health Sciences, Western University, London, ON, Canada
5 shared publications - Rashmi A. Kusurkar
Research in Education, Amsterdam UMC location Vrije Universiteit Amsterdam
5 shared publications - Melanie Anderson
University Health Network, Toronto, ON, Canada
4 shared publications Geoffrey Norman
Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
3 shared publications
Latest Publications
- Open Access
- 17 March 2026
- Published by Springer Nature in Medical Science Educator
- Vol. 36 (3) , 1195-1212
Testing that Makes You Think: Development of a Basic Science Test to Promote Future Clinical Learning
Introduction: Testing prior knowledge has been found to enhance learning new, related information. In this paper, we report on the development of a multiple-choice question (MCQ) test of basic science knowledge designed to elicit a particular kind of cognitive processing—distinctive processing—that, we argue, may be beneficial to future learning and clinical reasoning. Distinctive processing entails noticing both similarities and differences among concepts or entities. The recruitment and distinctive processing of prior basic science knowledge may optimally prepare learners to understand, remember and apply new, related clinical information. Methods: We developed two versions of a 19-item basic science MCQ. Both versions of the test had identical stems and correct response options; one version had very similar, plausible or “competitive” incorrect response options while the other version had less similar or “non-competitive” incorrect response options. We hypothesized that the “competitive” test version would elicit more distinctive processing than the “non-competitive” version. We conducted a concurrent verbal protocol to assess the response processes elicited by the two different test versions. Results: We found that the MCQ items designed with competitive or plausible incorrect answer options elicited more distinctive processing than MCQ items with less competitive or less plausible incorrect response options. Discussion: We showed that tests can be designed to promote cognitive processes that may be beneficial for future learning and that concurrent verbal protocols can be used to collect response process validity evidence for formative tests.
- 17 March 2026
- Published by Springer Nature in Medical Science Educator
- Vol. 36 (3) , 1195-1212
Open AccessTesting that Makes You Think: Development of a Basic Science Test to Promote Future Clinical Learning
Introduction: Testing prior knowledge has been found to enhance learning new, related information. In this paper, we report on the development of a multiple-choice question (MCQ) test of basic science knowledge designed to elicit a particular kind of cognitive processing—distinctive processing—that, we argue, may be beneficial to future learning and clinical reasoning. Distinctive processing entails noticing both similarities and differences among concepts or entities. The recruitment and distinctive processing of prior basic science knowledge may optimally prepare learners to understand, remember and apply new, related clinical information. Methods: We developed two versions of a 19-item basic science MCQ. Both versions of the test had identical stems and correct response options; one version had very similar, plausible or “competitive” incorrect response options while the other version had less similar or “non-competitive” incorrect response options. We hypothesized that the “competitive” test version would elicit more distinctive processing than the “non-competitive” version. We conducted a concurrent verbal protocol to assess the response processes elicited by the two different test versions. Results: We found that the MCQ items designed with competitive or plausible incorrect answer options elicited more distinctive processing than MCQ items with less competitive or less plausible incorrect response options. Discussion: We showed that tests can be designed to promote cognitive processes that may be beneficial for future learning and that concurrent verbal protocols can be used to collect response process validity evidence for formative tests.
- Open Access
- 11 April 2025
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 11, e64179
Motivation Theories and Constructs in Experimental Studies of Online Instruction: Systematic Review and Directed Content Analysis
Background: The motivational design of online instruction is critical in influencing learners’ motivation. Given the multifaceted and situated nature of motivation, educators need access to a range of evidence-based motivational design strategies that target different motivational constructs (eg, interest or confidence). Objective: This systematic review and directed content analysis aimed to catalog the motivational constructs targeted in experimental studies of online motivational design strategies in health professions education. Identifying which motivational constructs have been most frequently targeted by design strategies—and which remain under-studied—can offer valuable insights into potential areas for future research. Methods: Medline, Embase, Emcare, PsycINFO, ERIC, and Web of Science were searched from 1990 to August 2022. Studies were included if they compared online instructional design strategies intending to support a motivational construct (eg, interest) or motivation in general among learners in licensed health professions. Two team members independently screened and coded the studies, focusing on the motivational theories that researchers used and the motivational constructs targeted by their design strategies. Motivational constructs were coded into the following categories: intrinsic value beliefs, extrinsic value beliefs, competence and control beliefs, social connectedness, autonomy, and goals. Results: From 10,584 records, 46 studies were included. Half of the studies (n=23) tested strategies aimed at making instruction more interesting, enjoyable, and fun (n=23), while fewer studies tested strategies aimed at influencing extrinsic value beliefs (n=9), competence and control beliefs (n=6), social connectedness (n=4), or autonomy (n=2). A focus on intrinsic value beliefs was particularly evident in studies not informed by a theory of motivation. Conclusions: Most research in health professions education has focused on motivating learners by making online instruction more interesting, enjoyable, and fun. We recommend that future research expand this focus to include other motivational constructs, such as relevance, confidence, and autonomy. Investigating design strategies that influence these constructs would help generate a broader toolkit of strategies for educators to support learners’ motivation in online settings. Trial Registration: PROSPERO CRD42022359521; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022359521
- 11 April 2025
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 11, e64179
Open AccessMotivation Theories and Constructs in Experimental Studies of Online Instruction: Systematic Review and Directed Content Analysis
Background: The motivational design of online instruction is critical in influencing learners’ motivation. Given the multifaceted and situated nature of motivation, educators need access to a range of evidence-based motivational design strategies that target different motivational constructs (eg, interest or confidence). Objective: This systematic review and directed content analysis aimed to catalog the motivational constructs targeted in experimental studies of online motivational design strategies in health professions education. Identifying which motivational constructs have been most frequently targeted by design strategies—and which remain under-studied—can offer valuable insights into potential areas for future research. Methods: Medline, Embase, Emcare, PsycINFO, ERIC, and Web of Science were searched from 1990 to August 2022. Studies were included if they compared online instructional design strategies intending to support a motivational construct (eg, interest) or motivation in general among learners in licensed health professions. Two team members independently screened and coded the studies, focusing on the motivational theories that researchers used and the motivational constructs targeted by their design strategies. Motivational constructs were coded into the following categories: intrinsic value beliefs, extrinsic value beliefs, competence and control beliefs, social connectedness, autonomy, and goals. Results: From 10,584 records, 46 studies were included. Half of the studies (n=23) tested strategies aimed at making instruction more interesting, enjoyable, and fun (n=23), while fewer studies tested strategies aimed at influencing extrinsic value beliefs (n=9), competence and control beliefs (n=6), social connectedness (n=4), or autonomy (n=2). A focus on intrinsic value beliefs was particularly evident in studies not informed by a theory of motivation. Conclusions: Most research in health professions education has focused on motivating learners by making online instruction more interesting, enjoyable, and fun. We recommend that future research expand this focus to include other motivational constructs, such as relevance, confidence, and autonomy. Investigating design strategies that influence these constructs would help generate a broader toolkit of strategies for educators to support learners’ motivation in online settings. Trial Registration: PROSPERO CRD42022359521; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022359521
- 21 February 2025
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 100 (5) , 541-546
Artificial Intelligence Meets Holistic Review: Promises and Pitfalls of Automating the Medical Education Admissions Process
Holistic review has been widely adopted in medical education as a means of promoting equity in the application process and diversity in the medical workforce. Artificial intelligence (AI) is a rapidly emerging technology already having an impact on the medical school and residency application process as students and faculty alike increasingly turn to AI tools to automate some steps in the preparation and evaluation of application materials. While AI may have the potential to improve the holistic admissions process by increasing efficiency and adding some measure of standardization among reviewers, the authors caution that this promise does not come without certain pitfalls. AI models may introduce new sources of bias and amplify existing ones, which, when combined with a lack of transparency regarding their use in the admissions process, may perpetuate the very inequities that holistic review seeks to minimize. The authors call for the medical education community to establish clear regulations to govern the acceptable use of AI in the admissions process and for a principled adoption of AI tools in a way that is sustainable for applicants and reviewers in the future.
- 21 February 2025
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 100 (5) , 541-546
Artificial Intelligence Meets Holistic Review: Promises and Pitfalls of Automating the Medical Education Admissions Process
Holistic review has been widely adopted in medical education as a means of promoting equity in the application process and diversity in the medical workforce. Artificial intelligence (AI) is a rapidly emerging technology already having an impact on the medical school and residency application process as students and faculty alike increasingly turn to AI tools to automate some steps in the preparation and evaluation of application materials. While AI may have the potential to improve the holistic admissions process by increasing efficiency and adding some measure of standardization among reviewers, the authors caution that this promise does not come without certain pitfalls. AI models may introduce new sources of bias and amplify existing ones, which, when combined with a lack of transparency regarding their use in the admissions process, may perpetuate the very inequities that holistic review seeks to minimize. The authors call for the medical education community to establish clear regulations to govern the acceptable use of AI in the admissions process and for a principled adoption of AI tools in a way that is sustainable for applicants and reviewers in the future.
- Open Access
- 1 February 2025
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 30 (1) , 347
Correction to: Can all roads lead to competency? School levels effects in licensing examinations scores
- 1 February 2025
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 30 (1) , 347
Open AccessCorrection to: Can all roads lead to competency? School levels effects in licensing examinations scores
- Open Access
- 16 January 2025
- Published by Korea Health Personnel Licensing Examination Institute in Journal of Educational Evaluation for Health Professions
- Vol. 22, 5
Reliability and construct validation of the Blended Learning Usability Evaluation–Questionnaire with interprofessional clinicians in Canada: a methodological study
Purpose: To generate Cronbach’s alpha and further mixed methods construct validity evidence for the Blended Learning Usability Evaluation–Questionnaire (BLUE-Q).Methods: Forty interprofessional clinicians completed the BLUE-Q after finishing a 3-month long blended learning professional development program in Ontario, Canada. Reliability was assessed with Cronbach’s α for each of the 3 sections of the BLUE-Q and for all quantitative items together. Construct validity was evaluated through the Grand-Guillaume-Perrenoud et al. framework, which consists of 3 elements: congruence, convergence, and credibility. To compare quantitative and qualitative results, descriptive statistics, including means and standard deviations for each Likert scale item of the BLUE-Q were calculated.Results: Cronbach’s α was 0.95 for the pedagogical usability section, 0.85 for the synchronous modality section, 0.93 for the asynchronous modality section, and 0.96 for all quantitative items together. Mean ratings (with standard deviations) were 4.77 (0.506) for pedagogy, 4.64 (0.654) for synchronous learning, and 4.75 (0.536) for asynchronous learning. Of the 239 qualitative comments received, 178 were identified as substantive, of which 88% were considered congruent and 79% were considered convergent with the high means. Among all congruent responses, 69% were considered confirming statements and 31% were considered clarifying statements, suggesting appropriate credibility. Analysis of the clarifying statements assisted in identifying 5 categories of suggestions for program improvement.Conclusion: The BLUE-Q demonstrates high reliability and appropriate construct validity in the context of a blended learning program with interprofessional clinicians, making it a valuable tool for comprehensive program evaluation, quality improvement, and evaluative research in health professions education.
- 16 January 2025
- Published by Korea Health Personnel Licensing Examination Institute in Journal of Educational Evaluation for Health Professions
- Vol. 22, 5
Open AccessReliability and construct validation of the Blended Learning Usability Evaluation–Questionnaire with interprofessional clinicians in Canada: a methodological study
Purpose: To generate Cronbach’s alpha and further mixed methods construct validity evidence for the Blended Learning Usability Evaluation–Questionnaire (BLUE-Q).Methods: Forty interprofessional clinicians completed the BLUE-Q after finishing a 3-month long blended learning professional development program in Ontario, Canada. Reliability was assessed with Cronbach’s α for each of the 3 sections of the BLUE-Q and for all quantitative items together. Construct validity was evaluated through the Grand-Guillaume-Perrenoud et al. framework, which consists of 3 elements: congruence, convergence, and credibility. To compare quantitative and qualitative results, descriptive statistics, including means and standard deviations for each Likert scale item of the BLUE-Q were calculated.Results: Cronbach’s α was 0.95 for the pedagogical usability section, 0.85 for the synchronous modality section, 0.93 for the asynchronous modality section, and 0.96 for all quantitative items together. Mean ratings (with standard deviations) were 4.77 (0.506) for pedagogy, 4.64 (0.654) for synchronous learning, and 4.75 (0.536) for asynchronous learning. Of the 239 qualitative comments received, 178 were identified as substantive, of which 88% were considered congruent and 79% were considered convergent with the high means. Among all congruent responses, 69% were considered confirming statements and 31% were considered clarifying statements, suggesting appropriate credibility. Analysis of the clarifying statements assisted in identifying 5 categories of suggestions for program improvement.Conclusion: The BLUE-Q demonstrates high reliability and appropriate construct validity in the context of a blended learning program with interprofessional clinicians, making it a valuable tool for comprehensive program evaluation, quality improvement, and evaluative research in health professions education.
- Open Access
- 9 January 2025
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 14 (1) , 493-503
Many Minds, One Model: Exploring Decision Making of an Undergraduate Medicine Competency Committee Using the Construct of a Shared Mental Model
Introduction: Competency committees (CCs) are considered mandatory in competency-based medical education. There remains insufficient research to guide programs in optimizing the work of CCs especially in the undergraduate context. In order to address this gap, the functioning of an undergraduate CC is examined using the construct of a shared mental model (SMM) to explore factors and context that inform a holistic review of performance. Methods: A qualitative exploratory study was conducted. Using purposive sampling, 10 members of a Student Progress Committee (SPC) participated in 60-minute, semi-structured interviews (April 2022 to June 2023). An abductive thematic analysis approach generated themes which were then mapped onto a mental model construct. This heuristic helped construct and visualize the inner workings of a SMM as a holistic decision-making process that operates on manipulating multiple data inputs (quantitative and qualitative) in order to generate robust outcomes. Results: SPC members shared similar expectations of the task at hand while having multiple and conflicting perspectives about inputs important for decision making. Members grappled with what they perceived as a subjective process but agreed that having principles specific to holistic decision making can generate robust outcomes. Diversity of group membership was essential for minimizing member bias and group conformity in decision making. Discussion: This new understanding of how CCs operate at the undergraduate level can inform the SPC and guide its members in their quality improvement efforts and inform broader program-wide improvement, locally; moreover, it may contribute to the ongoing improvement of CCs in other settings.
- 9 January 2025
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 14 (1) , 493-503
Open AccessMany Minds, One Model: Exploring Decision Making of an Undergraduate Medicine Competency Committee Using the Construct of a Shared Mental Model
Introduction: Competency committees (CCs) are considered mandatory in competency-based medical education. There remains insufficient research to guide programs in optimizing the work of CCs especially in the undergraduate context. In order to address this gap, the functioning of an undergraduate CC is examined using the construct of a shared mental model (SMM) to explore factors and context that inform a holistic review of performance. Methods: A qualitative exploratory study was conducted. Using purposive sampling, 10 members of a Student Progress Committee (SPC) participated in 60-minute, semi-structured interviews (April 2022 to June 2023). An abductive thematic analysis approach generated themes which were then mapped onto a mental model construct. This heuristic helped construct and visualize the inner workings of a SMM as a holistic decision-making process that operates on manipulating multiple data inputs (quantitative and qualitative) in order to generate robust outcomes. Results: SPC members shared similar expectations of the task at hand while having multiple and conflicting perspectives about inputs important for decision making. Members grappled with what they perceived as a subjective process but agreed that having principles specific to holistic decision making can generate robust outcomes. Diversity of group membership was essential for minimizing member bias and group conformity in decision making. Discussion: This new understanding of how CCs operate at the undergraduate level can inform the SPC and guide its members in their quality improvement efforts and inform broader program-wide improvement, locally; moreover, it may contribute to the ongoing improvement of CCs in other settings.
- 5 December 2024
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 30 (1) , 37-52
Can all roads lead to competency? School levels effects in Licensing examinations scores
At the foundation of research concerned with professional training is the idea of an assumed causal chain between the policies and practices of education and the eventual behaviours of those that graduate these programs. In medicine, given the social accountability to ensure that teaching and learning gives way to a health human resource that is willing and able to provide the healthcare that patients and communities need, it is of critical importance to generate evidence regarding this causal relationship. One question that medical education scholars ask regularly is the degree to which the unique features of training programs and learning environments impact trainee achievement of the intended learning outcomes. To date, this evidence has been difficult to generate because data pertaining to learners is only rarely systematically brought together across institutions or periods of training. We describe new research which leverages an inter-institutional data-driven approach to investigate the influence of school-level factors on the licensing outcomes of medical students. Specifically, we bring together sociodemographic, admissions, and in-training assessment variables pertaining to medical trainee graduates at each of the six medical schools in Ontario, Canada into multilevel stepwise regression models that determine the degree of association between these variables and graduate performances on the Medical Council of Canada Qualifying Examinations (Part 1, n = 1097 observations; Part 2, n = 616 observations), established predictors of downstream physician performance. As part of this analysis, we include an anonymized school-level (School 1, School 2) independent variable in each of these models. Our results demonstrate that the largest variable associated with performance on both the first and second parts of the licensing examinations is prior academic achievement, notably clerkship performance. Ratings of biomedical knowledge were also significantly associated with the first examination, while clerkship OSCE scores and enrollment in a family medicine residency were significantly associated with the Part 2. Small significant school effects were realized in both models accounting for 4% and 2% of the variance realized in the first and second examinations, respectively. These findings highlight that school enrollment plays a minor role relative to individual student performance in influencing examination outcomes.
- 5 December 2024
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 30 (1) , 37-52
Can all roads lead to competency? School levels effects in Licensing examinations scores
At the foundation of research concerned with professional training is the idea of an assumed causal chain between the policies and practices of education and the eventual behaviours of those that graduate these programs. In medicine, given the social accountability to ensure that teaching and learning gives way to a health human resource that is willing and able to provide the healthcare that patients and communities need, it is of critical importance to generate evidence regarding this causal relationship. One question that medical education scholars ask regularly is the degree to which the unique features of training programs and learning environments impact trainee achievement of the intended learning outcomes. To date, this evidence has been difficult to generate because data pertaining to learners is only rarely systematically brought together across institutions or periods of training. We describe new research which leverages an inter-institutional data-driven approach to investigate the influence of school-level factors on the licensing outcomes of medical students. Specifically, we bring together sociodemographic, admissions, and in-training assessment variables pertaining to medical trainee graduates at each of the six medical schools in Ontario, Canada into multilevel stepwise regression models that determine the degree of association between these variables and graduate performances on the Medical Council of Canada Qualifying Examinations (Part 1, n = 1097 observations; Part 2, n = 616 observations), established predictors of downstream physician performance. As part of this analysis, we include an anonymized school-level (School 1, School 2) independent variable in each of these models. Our results demonstrate that the largest variable associated with performance on both the first and second parts of the licensing examinations is prior academic achievement, notably clerkship performance. Ratings of biomedical knowledge were also significantly associated with the first examination, while clerkship OSCE scores and enrollment in a family medicine residency were significantly associated with the Part 2. Small significant school effects were realized in both models accounting for 4% and 2% of the variance realized in the first and second examinations, respectively. These findings highlight that school enrollment plays a minor role relative to individual student performance in influencing examination outcomes.
- Open Access
- 10 July 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Motivation Theories and Constructs in Experimental Studies of Online Instruction: Systematic Review and Directed Content Analysis (Preprint)
BACKGROUND The motivational design of online instruction is critical in influencing learners’ motivation. Given the multifaceted and situated nature of motivation, educators need access to a range of evidence-based motivational design strategies that target different motivational constructs (eg, interest or confidence). OBJECTIVE This systematic review and directed content analysis aimed to catalog the motivational constructs targeted in experimental studies of online motivational design strategies in health professions education. Identifying which motivational constructs have been most frequently targeted by design strategies—and which remain under-studied—can offer valuable insights into potential areas for future research. METHODS Medline, Embase, Emcare, PsycINFO, ERIC, and Web of Science were searched from 1990 to August 2022. Studies were included if they compared online instructional design strategies intending to support a motivational construct (eg, interest) or motivation in general among learners in licensed health professions. Two team members independently screened and coded the studies, focusing on the motivational theories that researchers used and the motivational constructs targeted by their design strategies. Motivational constructs were coded into the following categories: intrinsic value beliefs, extrinsic value beliefs, competence and control beliefs, social connectedness, autonomy, and goals. RESULTS From 10,584 records, 46 studies were included. Half of the studies (n=23) tested strategies aimed at making instruction more interesting, enjoyable, and fun (n=23), while fewer studies tested strategies aimed at influencing extrinsic value beliefs (n=9), competence and control beliefs (n=6), social connectedness (n=4), or autonomy (n=2). A focus on intrinsic value beliefs was particularly evident in studies not informed by a theory of motivation. CONCLUSIONS Most research in health professions education has focused on motivating learners by making online instruction more interesting, enjoyable, and fun. We recommend that future research expand this focus to include other motivational constructs, such as relevance, confidence, and autonomy. Investigating design strategies that influence these constructs would help generate a broader toolkit of strategies for educators to support learners’ motivation in online settings. CLINICALTRIAL PROSPERO CRD42022359521; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022359521
- 10 July 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessMotivation Theories and Constructs in Experimental Studies of Online Instruction: Systematic Review and Directed Content Analysis (Preprint)
BACKGROUND The motivational design of online instruction is critical in influencing learners’ motivation. Given the multifaceted and situated nature of motivation, educators need access to a range of evidence-based motivational design strategies that target different motivational constructs (eg, interest or confidence). OBJECTIVE This systematic review and directed content analysis aimed to catalog the motivational constructs targeted in experimental studies of online motivational design strategies in health professions education. Identifying which motivational constructs have been most frequently targeted by design strategies—and which remain under-studied—can offer valuable insights into potential areas for future research. METHODS Medline, Embase, Emcare, PsycINFO, ERIC, and Web of Science were searched from 1990 to August 2022. Studies were included if they compared online instructional design strategies intending to support a motivational construct (eg, interest) or motivation in general among learners in licensed health professions. Two team members independently screened and coded the studies, focusing on the motivational theories that researchers used and the motivational constructs targeted by their design strategies. Motivational constructs were coded into the following categories: intrinsic value beliefs, extrinsic value beliefs, competence and control beliefs, social connectedness, autonomy, and goals. RESULTS From 10,584 records, 46 studies were included. Half of the studies (n=23) tested strategies aimed at making instruction more interesting, enjoyable, and fun (n=23), while fewer studies tested strategies aimed at influencing extrinsic value beliefs (n=9), competence and control beliefs (n=6), social connectedness (n=4), or autonomy (n=2). A focus on intrinsic value beliefs was particularly evident in studies not informed by a theory of motivation. CONCLUSIONS Most research in health professions education has focused on motivating learners by making online instruction more interesting, enjoyable, and fun. We recommend that future research expand this focus to include other motivational constructs, such as relevance, confidence, and autonomy. Investigating design strategies that influence these constructs would help generate a broader toolkit of strategies for educators to support learners’ motivation in online settings. CLINICALTRIAL PROSPERO CRD42022359521; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022359521
- 23 May 2024
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 29 (4) , 1393-1415
From admissions to licensure: education data associations from a multi-centre undergraduate medical education collaboration
This paper reports the findings of a Canada based multi-institutional study designed to investigate the relationships between admissions criteria, in-program assessments, and performance on licensing exams. The study’s objective is to provide valuable insights for improving educational practices across different institutions. Data were gathered from six medical schools: McMaster University, the Northern Ontario School of Medicine University, Queen’s University, University of Ottawa, University of Toronto, and Western University. The dataset includes graduates who undertook the Medical Council of Canada Qualifying Examination Part 1 (MCCQE1) between 2015 and 2017. The data were categorized into five distinct sections: demographic information as well as four matrices: admissions, course performance, objective structured clinical examination (OSCE), and clerkship performance. Common and unique variables were identified through an extensive consensus-building process. Hierarchical linear regression and a manual stepwise variable selection approach were used for analysis. Analyses were performed on data set encompassing graduates of all six medical schools as well as on individual data sets from each school. For the combined data set the final model estimated 32% of the variance in performance on licensing exams, highlighting variables such as Age at Admission, Sex, Biomedical Knowledge, the first post-clerkship OSCE, and a clerkship theta score. Individual school analysis explained 41–60% of the variance in MCCQE1 outcomes, with comparable variables to the analysis from of the combined data set identified as significant independent variables. Therefore, strongly emphasising the need for variety of high-quality assessment on the educational continuum. This study underscores the importance of sharing data to enable educational insights. This study also had its challenges when it came to the access and aggregation of data. As such we advocate for the establishment of a common framework for multi-institutional educational research, facilitating studies and evaluations across diverse institutions. This study demonstrates the scientific potential of collaborative data analysis in enhancing educational outcomes. It offers a deeper understanding of the factors influencing performance on licensure exams and emphasizes the need for addressing data gaps to advance multi-institutional research for educational improvements.
- 23 May 2024
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 29 (4) , 1393-1415
From admissions to licensure: education data associations from a multi-centre undergraduate medical education collaboration
This paper reports the findings of a Canada based multi-institutional study designed to investigate the relationships between admissions criteria, in-program assessments, and performance on licensing exams. The study’s objective is to provide valuable insights for improving educational practices across different institutions. Data were gathered from six medical schools: McMaster University, the Northern Ontario School of Medicine University, Queen’s University, University of Ottawa, University of Toronto, and Western University. The dataset includes graduates who undertook the Medical Council of Canada Qualifying Examination Part 1 (MCCQE1) between 2015 and 2017. The data were categorized into five distinct sections: demographic information as well as four matrices: admissions, course performance, objective structured clinical examination (OSCE), and clerkship performance. Common and unique variables were identified through an extensive consensus-building process. Hierarchical linear regression and a manual stepwise variable selection approach were used for analysis. Analyses were performed on data set encompassing graduates of all six medical schools as well as on individual data sets from each school. For the combined data set the final model estimated 32% of the variance in performance on licensing exams, highlighting variables such as Age at Admission, Sex, Biomedical Knowledge, the first post-clerkship OSCE, and a clerkship theta score. Individual school analysis explained 41–60% of the variance in MCCQE1 outcomes, with comparable variables to the analysis from of the combined data set identified as significant independent variables. Therefore, strongly emphasising the need for variety of high-quality assessment on the educational continuum. This study underscores the importance of sharing data to enable educational insights. This study also had its challenges when it came to the access and aggregation of data. As such we advocate for the establishment of a common framework for multi-institutional educational research, facilitating studies and evaluations across diverse institutions. This study demonstrates the scientific potential of collaborative data analysis in enhancing educational outcomes. It offers a deeper understanding of the factors influencing performance on licensure exams and emphasizes the need for addressing data gaps to advance multi-institutional research for educational improvements.
- 2 February 2024
- Published by Taylor & Francis in Medical Teacher
- Vol. 46 (4) , 471-485
Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship
Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.
- 2 February 2024
- Published by Taylor & Francis in Medical Teacher
- Vol. 46 (4) , 471-485
Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship
Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.