Factor Analysis Methods and Validity Evidence
- 1 August 2012
- journal article
- review article
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 87 (8) , 1060-1069
Abstract
Instrument development consistent with best practices is necessary for effective assessment and evaluation of learners and programs across the medical education continuum. The author explored the extent to which current factor analytic methods and other techniques for establishing validity are consistent with best practices. The author conducted electronic and hand searches of the English-language medical education literature published January 2006 through December 2010. To describe and assess current practices, she systematically abstracted reliability and validity evidence as well as factor analysis methods, data analysis, and reported evidence from instrument development articles reporting the application of exploratory factor analysis and principal component analysis. Sixty-two articles met eligibility criteria. They described 64 instruments and 95 factor analyses. Most studies provided at least one source of evidence based on test content. Almost all reported internal consistency, providing evidence based on internal structure. Evidence based on response process and relationships with other variables was reported less often, and evidence based on consequences of testing was not identified. Factor analysis findings suggest common method selection errors and critical omissions in reporting. Given the limited reliability and validity evidence provided for the reviewed instruments, educators should carefully consider the available supporting evidence before adopting and applying published instruments. Researchers should design for, test, and report additional evidence to strengthen the argument for reliability and validity of these measures for research and practice.Keywords
This publication has 88 references indexed in Scilit:
- Evaluation of Empathy Among Korean Medical Students: A Cross-Sectional Study Using the Korean Version of the Jefferson Scale of Physician EmpathyTeaching and Learning in Medicine, 2010
- Measurement of Empathy Among Japanese Medical Students: Psychometrics and Score Differences by Gender and Level of Medical EducationAcademic Medicine, 2009
- The Jefferson Scale of Physician Empathy: Preliminary Psychometrics and Group Comparisons in Italian PhysiciansAcademic Medicine, 2009
- A survey of student perceptions of team‐based learning in anatomy curriculum: Favorable views unrelated to gradesAnatomical Sciences Education, 2009
- Measuring Residents’ Perceived Preparedness and Skillfulness to Deliver Cross-cultural CareJournal of General Internal Medicine, 2009
- Measurement properties of the Inventory of Cognitive Bias in Medicine (ICBM)BMC Medical Informatics and Decision Making, 2008
- Validation of a Method for Assessing Resident Physicians’ Quality Improvement ProposalsJournal of General Internal Medicine, 2007
- Resident Perceptions of the Impact of Work Hour LimitationsJournal of General Internal Medicine, 2007
- Instruments for Evaluating Education in Evidence-Based PracticeJAMA, 2006
- Physician characteristics associated with proficiency in feedback skillsJournal of General Internal Medicine, 2006