Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries
Authors:
- Olakunle AlongeOAOlakunle Alonge
- Aditi RaoARAditi Rao
- Anna KalbarczykAKAnna Kalbarczyk
- Latifat IbisomiLILatifat Ibisomi
- Phyllis Dako-GyekePDPhyllis Dako-Gyeke
- Yodi MahendradhataYMYodi Mahendradhata
- Carlos Alberto RojasCRCarlos Alberto Rojas
- Choolwe JacobsCJChoolwe Jacobs
- Kwasi TorpeyKTKwasi Torpey
- Edwin Rolando Gonzalez MarulandaEMEdwin Rolando Gonzalez Marulanda
- Pascal LaunoisPLPascal Launois
- Mahnaz VahediMVMahnaz Vahedi
Open Access
Abstract
Introduction Methods and validated tools for evaluating the performance of competency-based implementation research (IR) training programmes in low–middle-income countries (LMICs) are lacking. In this study, we developed tools for assessing the performance of IR training programmes based on a framework of IR core competencies in LMICs. Methods We developed self-assessment and objective-assessment tools drawing on the IR competency framework. We used exploratory factor analyses and a one-parameter logistic model to establish construct validity and internal consistency of the tools drawing on a survey conducted in 2020 with 166 trainees before and after an index IR course across five universities and LMICs under the Special Program for Research and Training in Tropical Diseases postgraduate IR training scheme. We conducted key informant interviews (KII) with 20 trainees and five trainers to reflect on the usefulness of the tools and framework for guiding IR training in LMICs. Results Two 16-item tools for self-assessment of IR knowledge and self-efficacy and a 40-item objective assessment tool were developed. The factor loadings of items in the self-assessment tools were 0.65–0.87 with Cronbach’s alpha (α) of 0.97, and 0.77–0.91 with α of 0.98 for the IR knowledge and self-efficacy tools, respectively. The distribution of item difficulty in the objective-assessment tool was consistent before and after the index IR course. Pearson correlation coefficient (r) between the self-assessed and objectively assessed IR knowledge before the index IR course was low, r=0.27 (p value: Conclusion The IR competency framework and tools developed for assessing IR competencies and performance of IR training programmes in LMICs are reliable and valid. Self-assessment methods alone may not suffice to yield a valid assessment of performance in these settings.Keywords
Funding Information
- HRP Special Program of Research, Development, and Research Training in Human Reproduction (N/A)
- TDR Special Program for Research and Training in Tropical Diseases (N/A)
This publication has 44 references indexed in Scilit:
- Integrating competency-based education with a case-based or problem-based learning approach in online health sciencesAsia Pacific Education Review, 2020
- Competences for implementation science: what trainees need to learn and where they learn itAdvances in Health Sciences Education, 2020
- Implementation research: new imperatives and opportunities in global healthThe Lancet, 2018
- The effect of prequestions on learning from video presentations.Journal of Applied Research in Memory and Cognition, 2017
- The Testing Effect Is Alive and Well with Complex MaterialsEducational Psychology Review, 2015
- Systematic review of the Hawthorne effect: New concepts are needed to study research participation effectsJournal of Clinical Epidemiology, 2014
- Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study.Journal of Applied Research in Memory and Cognition, 2012
- Cognitive burden of survey questions and response times: A psycholinguistic experimentApplied Cognitive Psychology, 2009
- Factor analysis in the development and refinement of clinical assessment instruments.Psychological Assessment, 1995
- The effect of pre-testing on post-test performanceInstructional Science, 1973