Kelly L. Dore
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Kelly L. Dore has a total of 35 co-authors. Here are the top co-authors by number of shared publications:
Geoffrey R. Norman
Department of Clinical Epidemiology and Biostatistics, PERD, McMaster University, Hamilton, ON, L8S 4K1, Canada
5 shared publicationsHarold I. Reiter
Department of Oncology, PERD, McMaster University, Hamilton, ON, L8S 4K1, Canada
4 shared publicationsSharyn Kreuger
PERD, McMaster University, Hamilton, ON, L8S 4K1, Canada
4 shared publications- Geoff Norman
McMaster Health Education Research, Innovation and Theory (MERIT) Research Centre, McMaster University Hamilton Ontario Canada
2 shared publications - Jonathan Sherbino
Professor of Emergency Medicine, McMaster University Hamilton, Canada
2 shared publications Kevin W. Eva
2 shared publications
Latest Publications
- Open Access
- 29 September 2026
- Published by Springer Nature in BMC Medical Education
Do competency-aligned admissions assessments predict medical school awards? A validity argument using Kane’s framework
Background: Medical school admissions committees strive to use evidence-based, valid measures to select applicants who will succeed in training and excel in medical practice. Guided by Kane’s Validity Framework, this study builds a validity argument by examining predictive validity for four competency-based admissions assessments (GPA, MCAT, Casper SJT, and local MMI) in relation to academic and professional awards conferred during medical school training. Methods: A retrospective study was conducted on 644 graduates (2020–2022) from New York Medical College School of Medicine (NYMC SOM). Stepwise binomial logistic regression models assessed the association of the admissions assessments independently in predicting award attainment. Chi-square tests and contingency tables were used to examine the concordance between high admissions scores and award recipients. Results: MCAT scores were significantly associated with model improvement for Academic awards (OR = 1.43; 95% CI, 1.13–1.80, p < .001). Casper SJT and MMI scores were significantly associated with Service awards model improvement (OR = 1.24; 95% CI, 1.00-1.55, p = .010; OR = 1.39; 95% CI, 1.13–1.71, p = .002, respectively). Casper SJT scores were significantly associated with Clinical awards model improvement (OR, 1.46; 95% CI, 1.11–1.92, p < .003). Other metrics were non-significant. Conclusions: This study applies Kane’s framework to examine associations between admissions assessments and institutionally recognized award outcomes across academic and professional domains. The findings provide context-specific validity evidence relevant to the extrapolation inference and illustrate the potential value of considering broader outcome measures beyond traditional academic performance. Kane’s framework may be useful for institutions seeking to evaluate admissions assessments in relation to outcomes aligned with their educational goals.
- 29 September 2026
- Published by Springer Nature in BMC Medical Education
Open AccessDo competency-aligned admissions assessments predict medical school awards? A validity argument using Kane’s framework
Background: Medical school admissions committees strive to use evidence-based, valid measures to select applicants who will succeed in training and excel in medical practice. Guided by Kane’s Validity Framework, this study builds a validity argument by examining predictive validity for four competency-based admissions assessments (GPA, MCAT, Casper SJT, and local MMI) in relation to academic and professional awards conferred during medical school training. Methods: A retrospective study was conducted on 644 graduates (2020–2022) from New York Medical College School of Medicine (NYMC SOM). Stepwise binomial logistic regression models assessed the association of the admissions assessments independently in predicting award attainment. Chi-square tests and contingency tables were used to examine the concordance between high admissions scores and award recipients. Results: MCAT scores were significantly associated with model improvement for Academic awards (OR = 1.43; 95% CI, 1.13–1.80, p < .001). Casper SJT and MMI scores were significantly associated with Service awards model improvement (OR = 1.24; 95% CI, 1.00-1.55, p = .010; OR = 1.39; 95% CI, 1.13–1.71, p = .002, respectively). Casper SJT scores were significantly associated with Clinical awards model improvement (OR, 1.46; 95% CI, 1.11–1.92, p < .003). Other metrics were non-significant. Conclusions: This study applies Kane’s framework to examine associations between admissions assessments and institutionally recognized award outcomes across academic and professional domains. The findings provide context-specific validity evidence relevant to the extrapolation inference and illustrate the potential value of considering broader outcome measures beyond traditional academic performance. Kane’s framework may be useful for institutions seeking to evaluate admissions assessments in relation to outcomes aligned with their educational goals.
- 7 September 2019
- Published by Walter de Gruyter GmbH in Journal of Complementary and Integrative Medicine
- Vol. 17 (2)
“I am a healthcare practitioner”: A qualitative exploration of massage therapists’ professional identity
Background: A division has been described among massage therapists, some who identify as healthcare providers while others identify as service providers. The perceived division creates confusion about what it means to be a massage therapist. Objective: This qualitative study answered, “How do massage therapists in Ontario describe their professional identity?” Methods: Qualitative description (QD) was used and data were collected from 33 massage therapists using semi-structured interviews. Results: The resulting description of massage therapists’ identity in Ontario is the first of its kind. The identity described includes passion as professional motivation in practice, the importance of confidence and competence, a focus on the therapeutic relationship, individualized care, and patient empowerment, and a desire to be recognized for their role within the healthcare system. Conclusion: There is still much to be investigated about massage therapists’ identity. Future research will explore whether this description resonates with a larger sample of massage therapists in Ontario.
- 7 September 2019
- Published by Walter de Gruyter GmbH in Journal of Complementary and Integrative Medicine
- Vol. 17 (2)
“I am a healthcare practitioner”: A qualitative exploration of massage therapists’ professional identity
Background: A division has been described among massage therapists, some who identify as healthcare providers while others identify as service providers. The perceived division creates confusion about what it means to be a massage therapist. Objective: This qualitative study answered, “How do massage therapists in Ontario describe their professional identity?” Methods: Qualitative description (QD) was used and data were collected from 33 massage therapists using semi-structured interviews. Results: The resulting description of massage therapists’ identity in Ontario is the first of its kind. The identity described includes passion as professional motivation in practice, the importance of confidence and competence, a focus on the therapeutic relationship, individualized care, and patient empowerment, and a desire to be recognized for their role within the healthcare system. Conclusion: There is still much to be investigated about massage therapists’ identity. Future research will explore whether this description resonates with a larger sample of massage therapists in Ontario.
- Open Access
- 29 November 2018
- Published by Walter de Gruyter GmbH in Diagnosis
- Vol. 6 (3) , 259-268
Experienced physician descriptions of intuition in clinical reasoning: a typology
Background: Diagnostic intuition is a rapid, non-analytic, unconscious mode of reasoning. A small body of evidence points to the ubiquity of intuition, and its usefulness in generating diagnostic hypotheses and ascertaining severity of illness. Little is known about how experienced physicians understand this phenomenon, and how they work with it in clinical practice. Methods: Descriptions of how experienced physicians perceive their use of diagnostic intuition in clinical practice were elicited through interviews conducted with 30 physicians in emergency, internal and family medicine. Each participant was asked to share stories of diagnostic intuition, including times when intuition was both correct and incorrect. Multiple coders conducted descriptive analysis to analyze the salient aspects of these stories. Results: Physicians provided descriptions of what diagnostic intuition is, when it occurs and what type of activity it prompts. From stories of correct intuition, a typology of four different types of intuition was identified: Sick/Not Sick, Something Not Right, Frame-shifting and Abduction. Most physician accounts of diagnostic intuition linked this phenomenon to non-analytic reasoning and emphasized the importance of experience in developing a trustworthy sense of intuition that can be used to effectively engage analytic reasoning to evaluate clinical evidence. Conclusions: The participants recounted myriad stories of diagnostic intuition that alerted them to unusual diagnoses, previous diagnostic error or deleterious trajectories. While this qualitative study can offer no conclusions about the representativeness of these stories, it suggests that physicians perceive clinical intuition as beneficial for correcting and advancing diagnoses of both common and rare conditions.
- 29 November 2018
- Published by Walter de Gruyter GmbH in Diagnosis
- Vol. 6 (3) , 259-268
Open AccessExperienced physician descriptions of intuition in clinical reasoning: a typology
Background: Diagnostic intuition is a rapid, non-analytic, unconscious mode of reasoning. A small body of evidence points to the ubiquity of intuition, and its usefulness in generating diagnostic hypotheses and ascertaining severity of illness. Little is known about how experienced physicians understand this phenomenon, and how they work with it in clinical practice. Methods: Descriptions of how experienced physicians perceive their use of diagnostic intuition in clinical practice were elicited through interviews conducted with 30 physicians in emergency, internal and family medicine. Each participant was asked to share stories of diagnostic intuition, including times when intuition was both correct and incorrect. Multiple coders conducted descriptive analysis to analyze the salient aspects of these stories. Results: Physicians provided descriptions of what diagnostic intuition is, when it occurs and what type of activity it prompts. From stories of correct intuition, a typology of four different types of intuition was identified: Sick/Not Sick, Something Not Right, Frame-shifting and Abduction. Most physician accounts of diagnostic intuition linked this phenomenon to non-analytic reasoning and emphasized the importance of experience in developing a trustworthy sense of intuition that can be used to effectively engage analytic reasoning to evaluate clinical evidence. Conclusions: The participants recounted myriad stories of diagnostic intuition that alerted them to unusual diagnoses, previous diagnostic error or deleterious trajectories. While this qualitative study can offer no conclusions about the representativeness of these stories, it suggests that physicians perceive clinical intuition as beneficial for correcting and advancing diagnoses of both common and rare conditions.
- 23 October 2017
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 22 (5) , 1321-1322
Correction to: CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores
In re-examining the paper “CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores” published in AHSE ( 22(2), 327-336), we...
- 23 October 2017
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 22 (5) , 1321-1322
Correction to: CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores
In re-examining the paper “CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores” published in AHSE ( 22(2), 327-336), we...
- 21 November 2016
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 22 (2) , 327-336
CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores
Typically, only a minority of applicants to health professional training are invited to interview. However, pre-interview measures of cognitive skills predict for national licensure scores (Gauer et al. in Med Educ Online 21 2016) and subsequently licensure scores predict for performance in practice (Tamblyn et al. in JAMA 288(23): 3019–3026, 2002; Tamblyn et al. in JAMA 298(9):993–1001, 2007). Assessment of personal and professional characteristics, with the same psychometric rigour of measures of cognitive abilities, are needed upstream in the selection to health profession training programs. To fill that need, Computer-based Assessment for Sampling Personal characteristics (CASPer)—an on-line, video-based screening test—was created. In this paper, we examine the correlation between CASPer and Canadian national licensure examination outcomes in 109 doctors who took CASPer at the time of selection to medical school. Specifically, CASPer scores were correlated against performance on cognitive and ‘non-cognitive’ subsections of both the Medical Council of Canada Qualifying Examination (MCCQE) Parts I (end of medical school) and Part II (18 months into specialty training). Unlike most national licensure exams, MCCQE has specific subcomponents examining personal/professional qualities, providing a unique opportunity for comparison. The results demonstrated moderate predictive validity of CASPer to national licensure outcomes of personal/professional characteristics three to six years after admission to medical school. These types of disattenuated correlations (r = 0.3–0.5) are not otherwise predicted by traditional screening measures. These data support the ability of a computer-based strategy to screen applicants in a feasible, reliable test, which has now demonstrated predictive validity, lending evidence of its validation for medical school applicant selection.
- 21 November 2016
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 22 (2) , 327-336
CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores
Typically, only a minority of applicants to health professional training are invited to interview. However, pre-interview measures of cognitive skills predict for national licensure scores (Gauer et al. in Med Educ Online 21 2016) and subsequently licensure scores predict for performance in practice (Tamblyn et al. in JAMA 288(23): 3019–3026, 2002; Tamblyn et al. in JAMA 298(9):993–1001, 2007). Assessment of personal and professional characteristics, with the same psychometric rigour of measures of cognitive abilities, are needed upstream in the selection to health profession training programs. To fill that need, Computer-based Assessment for Sampling Personal characteristics (CASPer)—an on-line, video-based screening test—was created. In this paper, we examine the correlation between CASPer and Canadian national licensure examination outcomes in 109 doctors who took CASPer at the time of selection to medical school. Specifically, CASPer scores were correlated against performance on cognitive and ‘non-cognitive’ subsections of both the Medical Council of Canada Qualifying Examination (MCCQE) Parts I (end of medical school) and Part II (18 months into specialty training). Unlike most national licensure exams, MCCQE has specific subcomponents examining personal/professional qualities, providing a unique opportunity for comparison. The results demonstrated moderate predictive validity of CASPer to national licensure outcomes of personal/professional characteristics three to six years after admission to medical school. These types of disattenuated correlations (r = 0.3–0.5) are not otherwise predicted by traditional screening measures. These data support the ability of a computer-based strategy to screen applicants in a feasible, reliable test, which has now demonstrated predictive validity, lending evidence of its validation for medical school applicant selection.
- 4 November 2016
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 22 (2) , 565-572
Widening perspectives: reframing the way we research selection
- 4 November 2016
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 22 (2) , 565-572
Widening perspectives: reframing the way we research selection
- Open Access
- 1 February 2014
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 89 (2) , 277-284
The Etiology of Diagnostic Errors
Purpose Diagnostic errors are thought to arise from cognitive biases associated with System 1 reasoning, which is rapid and unconscious. The primary hypothesis of this study was that the instruction to be slow and thorough will have no advantage in diagnostic accuracy over the instruction to proceed rapidly. Method Participants were second-year residents who volunteered after they had taken the Medical Council of Canada (MCC) Qualifying Examination Part II. Participants were tested at three Canadian medical schools (McMaster, Ottawa, and McGill) in 2010 (n = 96) and 2011 (n = 108). The intervention consisted of 20 computer-based internal medicine cases, with instructions either (1) to be as quick as possible but not make mistakes (the Speed cohort, 2010), or (2) to be careful, thorough, and reflective (the Reflect cohort, 2011). The authors examined accuracy scores on the 20 cases, time taken to diagnose cases, and MCC examination performance. Results Overall accuracy in the Speed condition was 44.5%, and in the Reflect condition was 45.0%; this was not significant. The Speed cohort took an average of 69 seconds per case versus 89 seconds for the Reflect cohort (P < .001). In both cohorts, cases diagnosed incorrectly took an average of 17 seconds longer than cases diagnosed correctly. Diagnostic accuracy was moderately correlated with performance on both written and problem-solving components of the MCC licensure examination and inversely correlated with time. Conclusions The study demonstrates that simply encouraging slowing down and increasing attention to analytical thinking is insufficient to increase diagnostic accuracy.
- 1 February 2014
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 89 (2) , 277-284
Open AccessThe Etiology of Diagnostic Errors
Purpose Diagnostic errors are thought to arise from cognitive biases associated with System 1 reasoning, which is rapid and unconscious. The primary hypothesis of this study was that the instruction to be slow and thorough will have no advantage in diagnostic accuracy over the instruction to proceed rapidly. Method Participants were second-year residents who volunteered after they had taken the Medical Council of Canada (MCC) Qualifying Examination Part II. Participants were tested at three Canadian medical schools (McMaster, Ottawa, and McGill) in 2010 (n = 96) and 2011 (n = 108). The intervention consisted of 20 computer-based internal medicine cases, with instructions either (1) to be as quick as possible but not make mistakes (the Speed cohort, 2010), or (2) to be careful, thorough, and reflective (the Reflect cohort, 2011). The authors examined accuracy scores on the 20 cases, time taken to diagnose cases, and MCC examination performance. Results Overall accuracy in the Speed condition was 44.5%, and in the Reflect condition was 45.0%; this was not significant. The Speed cohort took an average of 69 seconds per case versus 89 seconds for the Reflect cohort (P < .001). In both cohorts, cases diagnosed incorrectly took an average of 17 seconds longer than cases diagnosed correctly. Diagnostic accuracy was moderately correlated with performance on both written and problem-solving components of the MCC licensure examination and inversely correlated with time. Conclusions The study demonstrates that simply encouraging slowing down and increasing attention to analytical thinking is insufficient to increase diagnostic accuracy.
- 1 January 2012
- Published by American Psychological Association (APA) in Journal of Experimental Psychology: Applied
- Vol. 18 (1) , 109-125
Influence of familiar features on diagnosis: Instantiated features in an applied setting.
Medical diagnosis can be viewed as a categorization task. There are two mechanisms whereby humans make categorical judgments: "analytical reasoning," based on explicit consideration of features and "nonanalytical reasoning," an unconscious holistic process of matching against prior exemplars. However, there is evidence that prior experience can also operate at the level of individual "instantiated" features (Brooks & Hannah, 2006). The present studies examined instantiated features in medical diagnosis. Four "pseudopsychiatric" conditions, each described by four characteristic features, were taught to undergraduate psychology students. They practiced on additional cases, then were tested on new cases with features from two conditions. In Experiment 1, diagnoses associated with familiar features presented one or three times during practice were assigned a higher probability than those with novel features. Experiment 2 showed that the impact of feature frequency was dependent on its consistency with the case diagnosis. Experiment 3 showed that the effect of feature familiarity was not confined to cases with two equiprobable diagnoses. Experiment 4 showed that the effect remained after a 24 hour delay. These four studies demonstrated that features seen in practice have a greater influence on diagnosis than novel synonyms. In fact, seeing a feature once within the appropriate context (a patient case in which it is a member of the primary diagnosis) was sufficient to form a diagnostic association equivalent to instantiations seen four times in a different context. The results of these studies have implications for theories of categorization and for teaching clinical reasoning.
- 1 January 2012
- Published by American Psychological Association (APA) in Journal of Experimental Psychology: Applied
- Vol. 18 (1) , 109-125
Influence of familiar features on diagnosis: Instantiated features in an applied setting.
Medical diagnosis can be viewed as a categorization task. There are two mechanisms whereby humans make categorical judgments: "analytical reasoning," based on explicit consideration of features and "nonanalytical reasoning," an unconscious holistic process of matching against prior exemplars. However, there is evidence that prior experience can also operate at the level of individual "instantiated" features (Brooks & Hannah, 2006). The present studies examined instantiated features in medical diagnosis. Four "pseudopsychiatric" conditions, each described by four characteristic features, were taught to undergraduate psychology students. They practiced on additional cases, then were tested on new cases with features from two conditions. In Experiment 1, diagnoses associated with familiar features presented one or three times during practice were assigned a higher probability than those with novel features. Experiment 2 showed that the impact of feature frequency was dependent on its consistency with the case diagnosis. Experiment 3 showed that the effect of feature familiarity was not confined to cases with two equiprobable diagnoses. Experiment 4 showed that the effect remained after a 24 hour delay. These four studies demonstrated that features seen in practice have a greater influence on diagnosis than novel synonyms. In fact, seeing a feature once within the appropriate context (a patient case in which it is a member of the primary diagnosis) was sufficient to form a diagnostic association equivalent to instantiations seen four times in a different context. The results of these studies have implications for theories of categorization and for teaching clinical reasoning.
- 1 October 2010
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 85, S60-S63
The Reliability and Acceptability of the Multiple Mini-Interview as a Selection Instrument for Postgraduate Admissions
Background The Multiple Mini-Interview (MMI) is useful in selecting undergraduate medical trainees. Postgraduate applicant pools have smaller numbers of more homogeneous candidates that must be actively recruited while being assessed. This paper reports on the MMI's use in assessing residency candidates. Method Canadian and international medical graduates to three residency programs—obstetrics–gynecology and pediatrics (McMaster University) and internal medicine (University of Alberta)—underwent the MMI for residency selection (n = 484) in 2008 and 2009. Reliability was determined and candidates and interviewers completed an exit survey assessing acceptability. Results Overall reliability of the MMI was acceptable, ranging from 0.55 to 0.72. Using 10 stations would increase reliability to 0.64–0.79. Eighty-eight percent of candidates believed they could accurately portray themselves, while 90% of interviewers believed they could reasonably judge candidates' abilities. Conclusions The MMI provides a reliable way to assess residency candidates that is acceptable to both candidates and assessors across a variety of programs.
- 1 October 2010
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 85, S60-S63
The Reliability and Acceptability of the Multiple Mini-Interview as a Selection Instrument for Postgraduate Admissions
Background The Multiple Mini-Interview (MMI) is useful in selecting undergraduate medical trainees. Postgraduate applicant pools have smaller numbers of more homogeneous candidates that must be actively recruited while being assessed. This paper reports on the MMI's use in assessing residency candidates. Method Canadian and international medical graduates to three residency programs—obstetrics–gynecology and pediatrics (McMaster University) and internal medicine (University of Alberta)—underwent the MMI for residency selection (n = 484) in 2008 and 2009. Reliability was determined and candidates and interviewers completed an exit survey assessing acceptability. Results Overall reliability of the MMI was acceptable, ranging from 0.55 to 0.72. Using 10 stations would increase reliability to 0.64–0.79. Eighty-eight percent of candidates believed they could accurately portray themselves, while 90% of interviewers believed they could reasonably judge candidates' abilities. Conclusions The MMI provides a reliable way to assess residency candidates that is acceptable to both candidates and assessors across a variety of programs.
- 1 October 2009
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 84 (Supplement) , S9-S12
Extending the Interview to All Medical School Candidates—Computer-Based Multiple Sample Evaluation of Noncognitive Skills (CMSENS)
Most medical school candidates are excluded without benefit of noncognitive skills assessment. Is development of a noncognitive preinterview screening test that correlates with the well-validated Multiple Mini-Interview (MMI) possible? Study 1: 110 medical school candidates completed MMI and Computer-based Multiple Sample Evaluation of Noncognitive Skills (CMSENS)—eight 1-minute video-based scenarios and four self-descriptive questions, with short-answer-response format. Seventy-eight responses were audiotaped, 32 typewritten; all were scored by two independent raters. Study 2: 167 candidates completed CMSENS—eight videos, six self-descriptive questions, typewritten responses only, scored by two raters; 88 of 167 underwent the MMI. Results for overall test generalizability, interrater reliability, and correlation with MMI, respectively, were, for Study 1, audio-responders: 0.86, 0.82, 0.15; typewritten-responders: 0.72, 0.81, 0.51; and for Study 2, 0.83, 0.95, 0.46 (correlation with disattenuation was 0.60). Strong psychometric properties, including MMI correlation, of CMSENS warrant investigation into future widespread implementation as a preinterview noncognitive screening test.
- 1 October 2009
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 84 (Supplement) , S9-S12
Extending the Interview to All Medical School Candidates—Computer-Based Multiple Sample Evaluation of Noncognitive Skills (CMSENS)
Most medical school candidates are excluded without benefit of noncognitive skills assessment. Is development of a noncognitive preinterview screening test that correlates with the well-validated Multiple Mini-Interview (MMI) possible? Study 1: 110 medical school candidates completed MMI and Computer-based Multiple Sample Evaluation of Noncognitive Skills (CMSENS)—eight 1-minute video-based scenarios and four self-descriptive questions, with short-answer-response format. Seventy-eight responses were audiotaped, 32 typewritten; all were scored by two independent raters. Study 2: 167 candidates completed CMSENS—eight videos, six self-descriptive questions, typewritten responses only, scored by two raters; 88 of 167 underwent the MMI. Results for overall test generalizability, interrater reliability, and correlation with MMI, respectively, were, for Study 1, audio-responders: 0.86, 0.82, 0.15; typewritten-responders: 0.72, 0.81, 0.51; and for Study 2, 0.83, 0.95, 0.46 (correlation with disattenuation was 0.60). Strong psychometric properties, including MMI correlation, of CMSENS warrant investigation into future widespread implementation as a preinterview noncognitive screening test.