Alina Smirnova
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Alina Smirnova has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
- Kiki M. J. M. H. Lombarts
Department of Medical Psychology Professional Performance & Compassionate Care Research group Amsterdam UMC University of Amsterdam Meibergdreef 9 1105 AZ Amsterdam The Netherlands
4 shared publications - Abigail Md Schuh
Department of Pediatrics, Medical College of Wisconsin, Milwaukee, WI
3 shared publications - Daniel J. Schumacher
Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
3 shared publications - Michael A. Barone
President and CEO of the American Board of Pediatrics, US
3 shared publications - Saad Chahine
Faculty of Education, Queen’s University, Canada
3 shared publications - Stefanie Susane Sebok‐Syer
Department of Emergency Medicine Stanford University Palo Alto California USA
3 shared publications
Latest Publications
- Open Access
- 9 January 2025
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 14 (1) , 1-8
Introducing the Next Era in Assessment
In this introduction, the guest editors of the “Next Era in Assessment” special collection frame the invited papers by envisioning a next era in assessment of medical education, based on ideas developed during a summit that convened professional and educational leaders and scholars. The authors posit that the next era of assessment will focus unambiguously on serving patients and the health of society, reflect its sociocultural context, and support learners’ longitudinal growth and development. As such, assessment will be characterized as transformational, development-oriented and socially accountable. The authors introduce the papers in this special collection, which represent elements of a roadmap towards the next era in assessment by exploring several foundational considerations that will make the next era successful. These include the equally important issues of (1) focusing on accountability, trust and power in assessment, (2) addressing implementation and contextualization of assessment systems, (3) optimizing the use of technology in assessment, (4) establishing infrastructure for data sharing and data storage, (5) developing a vocabulary around emerging sources of assessment data, and (6) reconceptualizing validity around patient care and learner equity. Attending to these priority areas will help leaders create authentic assessment systems that are responsive to learners’ and society’s needs, while reaping the full promise of competency-based medical education (CBME) as well as emerging data science and artificial intelligence technologies.
- 9 January 2025
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 14 (1) , 1-8
Open AccessIntroducing the Next Era in Assessment
In this introduction, the guest editors of the “Next Era in Assessment” special collection frame the invited papers by envisioning a next era in assessment of medical education, based on ideas developed during a summit that convened professional and educational leaders and scholars. The authors posit that the next era of assessment will focus unambiguously on serving patients and the health of society, reflect its sociocultural context, and support learners’ longitudinal growth and development. As such, assessment will be characterized as transformational, development-oriented and socially accountable. The authors introduce the papers in this special collection, which represent elements of a roadmap towards the next era in assessment by exploring several foundational considerations that will make the next era successful. These include the equally important issues of (1) focusing on accountability, trust and power in assessment, (2) addressing implementation and contextualization of assessment systems, (3) optimizing the use of technology in assessment, (4) establishing infrastructure for data sharing and data storage, (5) developing a vocabulary around emerging sources of assessment data, and (6) reconceptualizing validity around patient care and learner equity. Attending to these priority areas will help leaders create authentic assessment systems that are responsive to learners’ and society’s needs, while reaping the full promise of competency-based medical education (CBME) as well as emerging data science and artificial intelligence technologies.
- Open Access
- 20 November 2024
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 13 (1) , 553-560
Digital Evidence: Revisiting Assumptions at the Intersection of Technology and Assessment
The increasing use of technology in health care and health professions education is an invitation to examine how digital sources of evidence are used in making assessment claims. In this paper, we describe how four sets of terms—primary and secondary data; structured and unstructured data; development and use; and deterministic and generative—can aid in examining how data from digital sources are used in evaluating what learners know and can do. Drawing on multiple examples, this paper shows how the four sets of terms can help both developers and users of technology-based assessment systems.
- 20 November 2024
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 13 (1) , 553-560
Open AccessDigital Evidence: Revisiting Assumptions at the Intersection of Technology and Assessment
The increasing use of technology in health care and health professions education is an invitation to examine how digital sources of evidence are used in making assessment claims. In this paper, we describe how four sets of terms—primary and secondary data; structured and unstructured data; development and use; and deterministic and generative—can aid in examining how data from digital sources are used in evaluating what learners know and can do. Drawing on multiple examples, this paper shows how the four sets of terms can help both developers and users of technology-based assessment systems.
- Open Access
- 7 October 2024
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 13 (1) , 486-495
Sharing Is Caring: Helping Institutions and Health Organizations Leverage Data for Educational Improvement
Perspectives on Medical Education’s mission is to support and enrich collaborative scholarship between education researchers and clinical educators, and to advance new knowledge regarding clinical education practices. The journal is the official journal of the The Netherlands Association of Medical Education (NVMO), which sponsors the journal as an Open Access journal that is free to read immediately upon publication. To publish in the journal, authors are required to pay an author processing charge of £996 upon publication. The journal is indexed in key databases, including MEDLINE, Web of Science, Google Scholar, and Scopus and has received the DOAJ seal. In 2023, the journal was awarded an impact factor of 4.8. The journal is overseen by the Editor-in-Chief Professor Erik Driessen, PhD. He is provided by guidance by an international editorial board of leading scholars with members representing the different parental disciplines in the field of medical education, e.g. clinicians, social scientists, biomedical scientists, statisticians, and linguists.
- 7 October 2024
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 13 (1) , 486-495
Open AccessSharing Is Caring: Helping Institutions and Health Organizations Leverage Data for Educational Improvement
Perspectives on Medical Education’s mission is to support and enrich collaborative scholarship between education researchers and clinical educators, and to advance new knowledge regarding clinical education practices. The journal is the official journal of the The Netherlands Association of Medical Education (NVMO), which sponsors the journal as an Open Access journal that is free to read immediately upon publication. To publish in the journal, authors are required to pay an author processing charge of £996 upon publication. The journal is indexed in key databases, including MEDLINE, Web of Science, Google Scholar, and Scopus and has received the DOAJ seal. In 2023, the journal was awarded an impact factor of 4.8. The journal is overseen by the Editor-in-Chief Professor Erik Driessen, PhD. He is provided by guidance by an international editorial board of leading scholars with members representing the different parental disciplines in the field of medical education, e.g. clinicians, social scientists, biomedical scientists, statisticians, and linguists.
- Open Access
- 22 January 2024
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 13 (1) , 12-23
The Next Era of Assessment: Building a Trustworthy Assessment System
Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.
- 22 January 2024
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 13 (1) , 12-23
Open AccessThe Next Era of Assessment: Building a Trustworthy Assessment System
Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.
- 8 November 2022
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 98 (3) , 367-375
Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents’ Performance in Pediatric Emergency Medicine
Purpose: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents’ clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). Method: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children’s Hospital Medical Center, a quaternary children’s hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents’ rank after case-mix adjustments were also identified. Results: 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0–1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2–21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations (r = 0.90–0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents’ absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. Conclusions: For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents’ clinical performance and identify opportunities for quality improvement.
- 8 November 2022
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 98 (3) , 367-375
Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents’ Performance in Pediatric Emergency Medicine
Purpose: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents’ clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). Method: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children’s Hospital Medical Center, a quaternary children’s hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents’ rank after case-mix adjustments were also identified. Results: 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0–1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2–21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations (r = 0.90–0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents’ absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. Conclusions: For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents’ clinical performance and identify opportunities for quality improvement.
- Open Access
- 26 August 2022
- Published by The Association of Faculties of Medicine of Canada in Canadian Medical Education Journal
- Vol. 13 (4) , 23-29
Licensing exams in Canada: a closer look at the validity of the MCCQE Part II
The Medical Council of Canada Qualifying Exam (MCCQE) Part II aims to protect societal interests through examining recently graduated physicians using clinical scenarios with standardized patients. This position paper debates the role of the MCCQE Part II in the national licensing of physicians in Canada by focusing on the consequential validity evidence of this exam and considering future directions through discussing contemporary developments in high stakes examinations. Specifically, this paper compares both MCCQE Part I and Part II in their ability to predict future practice patterns of physicians and generalizability across specialties. In weighing up the evidence this paper considers commonly used counterarguments as well as the financial implications of this exam for both the candidates and the MCC. Finally, it concludes by providing recommendations for future licensing of physicians in Canada. The available consequential validity evidence for MCCQE Part II is limited. Though still limited, MCCQE Part I has more robust evidence that it is a better predictor of future practice patterns compared to with Part II. Combined with a lack of evidence that national licensing examinations lead to graduation of substandard doctors or an improvement of care, and the shift away from assessment of learning towards assessment for learning, the maximum impact of the MCC on safeguarding public’s interests will lie in working closely with residency programs and specialty colleges to facilitate a robust assessment program of essential competencies and clinical skills during residency training and specialty certification.
- 26 August 2022
- Published by The Association of Faculties of Medicine of Canada in Canadian Medical Education Journal
- Vol. 13 (4) , 23-29
Open AccessLicensing exams in Canada: a closer look at the validity of the MCCQE Part II
The Medical Council of Canada Qualifying Exam (MCCQE) Part II aims to protect societal interests through examining recently graduated physicians using clinical scenarios with standardized patients. This position paper debates the role of the MCCQE Part II in the national licensing of physicians in Canada by focusing on the consequential validity evidence of this exam and considering future directions through discussing contemporary developments in high stakes examinations. Specifically, this paper compares both MCCQE Part I and Part II in their ability to predict future practice patterns of physicians and generalizability across specialties. In weighing up the evidence this paper considers commonly used counterarguments as well as the financial implications of this exam for both the candidates and the MCC. Finally, it concludes by providing recommendations for future licensing of physicians in Canada. The available consequential validity evidence for MCCQE Part II is limited. Though still limited, MCCQE Part I has more robust evidence that it is a better predictor of future practice patterns compared to with Part II. Combined with a lack of evidence that national licensing examinations lead to graduation of substandard doctors or an improvement of care, and the shift away from assessment of learning towards assessment for learning, the maximum impact of the MCC on safeguarding public’s interests will lie in working closely with residency programs and specialty colleges to facilitate a robust assessment program of essential competencies and clinical skills during residency training and specialty certification.
- 1 March 2019
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 94 (3) , 419-426
The Association Between Residency Learning Climate and Inpatient Care Experience in Clinical Teaching Departments in the Netherlands
Purpose: To examine the association between residency learning climate and inpatient care experience. Method: The authors analyzed 1,201 evaluations of the residency learning climate (using the Dutch Residency Educational Climate Test questionnaire) and 6,689 evaluations of inpatient care experience (using the Consumer Quality Index Inpatient Hospital Care questionnaire) from 86 departments across 15 specialties in 18 hospitals in the Netherlands between 2013 and 2014. The authors used linear hierarchical panel analyses to study the associations between departments’ overall and subscale learning climate scores and inpatient care experience global ratings and subscale scores, controlling for respondent- and department-level characteristics and correcting for multiple testing. Results: Overall learning climate was not associated with global department ratings ( b = 0.03; 95% confidence interval −0.17 to 0.23) but was positively associated with specific inpatient care experience domains, including communication with doctors ( b = 0.11; 0.02 to 0.20) and feeling of safety ( b = 0.09; 0.01 to 0.17). Coaching and assessment was positively associated with communication with doctors ( b = 0.22; 0.08 to 0.37) and explanation of treatment ( b = 0.22; 0.08 to 0.36). Formal education was negatively associated with pain management ( b = −0.16; −0.26 to −0.05), while peer collaboration was positively associated with pain management ( b = 0.14; 0.03 to 0.24). Conclusions: Optimizing the clinical learning environment is an important step toward ensuring high-quality residency training and patient care. These findings could help clinical teaching departments address those aspects of the learning environment that directly affect patient care.
- 1 March 2019
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 94 (3) , 419-426
The Association Between Residency Learning Climate and Inpatient Care Experience in Clinical Teaching Departments in the Netherlands
Purpose: To examine the association between residency learning climate and inpatient care experience. Method: The authors analyzed 1,201 evaluations of the residency learning climate (using the Dutch Residency Educational Climate Test questionnaire) and 6,689 evaluations of inpatient care experience (using the Consumer Quality Index Inpatient Hospital Care questionnaire) from 86 departments across 15 specialties in 18 hospitals in the Netherlands between 2013 and 2014. The authors used linear hierarchical panel analyses to study the associations between departments’ overall and subscale learning climate scores and inpatient care experience global ratings and subscale scores, controlling for respondent- and department-level characteristics and correcting for multiple testing. Results: Overall learning climate was not associated with global department ratings ( b = 0.03; 95% confidence interval −0.17 to 0.23) but was positively associated with specific inpatient care experience domains, including communication with doctors ( b = 0.11; 0.02 to 0.20) and feeling of safety ( b = 0.09; 0.01 to 0.17). Coaching and assessment was positively associated with communication with doctors ( b = 0.22; 0.08 to 0.37) and explanation of treatment ( b = 0.22; 0.08 to 0.36). Formal education was negatively associated with pain management ( b = −0.16; −0.26 to −0.05), while peer collaboration was positively associated with pain management ( b = 0.14; 0.03 to 0.24). Conclusions: Optimizing the clinical learning environment is an important step toward ensuring high-quality residency training and patient care. These findings could help clinical teaching departments address those aspects of the learning environment that directly affect patient care.
- Open Access
- 20 February 2017
- Published by Wiley in Health Expectations
- Vol. 20 (5) , 1041-1048
Closing the patient experience chasm: A two‐level validation of the Consumer Quality Index Inpatient Hospital Care
Background: Evaluation of patients’ health care experiences is central to measuring patient‐centred care. However, different instruments tend to be used at the hospital or departmental level but rarely both, leading to a lack of standardization of patient experience measures.Objective: To validate the Consumer Quality Index (CQI) Inpatient Hospital Care for use on both department and hospital levels.Design: Using cross‐sectional observational data, we investigated the internal validity of the questionnaire using confirmatory factor analyses (CFA), and the generalizability of the questionnaire for use at the department and hospital levels using generalizability theory.Setting and participants: 22924 adults hospitalized for ≥24 hours between 1 January 2013 and 31 December 2014 in 23 Dutch hospitals (515 department evaluations).Main variable: CQIInpatient Hospital Care questionnaire.Results: CFAresults showed a good fit on individual level (CFI=0.96,TLI=0.95,RMSEA=0.04), which was comparable between specialties. When scores were aggregated to the department level, the fit was less desirable (CFI=0.83,TLI=0.81,RMSEA=0.06), and there was a significant overlap betweencommunication with doctorsandexplanation of treatmentsubscales. Departments and hospitals explained ≤5% of total variance in subscale scores. In total, 4‐8 departments and 50 respondents per department are needed to reliably evaluate subscales rated on a 4‐point scale, and 10 departments with 100‐150 respondents per department for binary subscales.Discussion and conclusions: TheCQIInpatient Hospital Care is a valid and reliable questionnaire to evaluate inpatient experiences in Dutch hospitals provided sufficient sampling is done. Results can facilitate meaningful comparisons and guide quality improvement activities in individual departments and hospitals.
- 20 February 2017
- Published by Wiley in Health Expectations
- Vol. 20 (5) , 1041-1048
Open AccessClosing the patient experience chasm: A two‐level validation of the Consumer Quality Index Inpatient Hospital Care
Background: Evaluation of patients’ health care experiences is central to measuring patient‐centred care. However, different instruments tend to be used at the hospital or departmental level but rarely both, leading to a lack of standardization of patient experience measures.Objective: To validate the Consumer Quality Index (CQI) Inpatient Hospital Care for use on both department and hospital levels.Design: Using cross‐sectional observational data, we investigated the internal validity of the questionnaire using confirmatory factor analyses (CFA), and the generalizability of the questionnaire for use at the department and hospital levels using generalizability theory.Setting and participants: 22924 adults hospitalized for ≥24 hours between 1 January 2013 and 31 December 2014 in 23 Dutch hospitals (515 department evaluations).Main variable: CQIInpatient Hospital Care questionnaire.Results: CFAresults showed a good fit on individual level (CFI=0.96,TLI=0.95,RMSEA=0.04), which was comparable between specialties. When scores were aggregated to the department level, the fit was less desirable (CFI=0.83,TLI=0.81,RMSEA=0.06), and there was a significant overlap betweencommunication with doctorsandexplanation of treatmentsubscales. Departments and hospitals explained ≤5% of total variance in subscale scores. In total, 4‐8 departments and 50 respondents per department are needed to reliably evaluate subscales rated on a 4‐point scale, and 10 departments with 100‐150 respondents per department for binary subscales.Discussion and conclusions: TheCQIInpatient Hospital Care is a valid and reliable questionnaire to evaluate inpatient experiences in Dutch hospitals provided sufficient sampling is done. Results can facilitate meaningful comparisons and guide quality improvement activities in individual departments and hospitals.