Validity and Reliability of Portfolio Assessment of Student Competence in Two Dental School Populations: A Four‐Year Study
- 1 May 2014
- journal article
- research article
- Published by Wiley in Journal of Dental Education
- Vol. 78 (5) , 657-667
Abstract
The purpose of this study was to empirically investigate the validity and reliability of portfolio assessment in two U.S. dental schools using a unified framework for validity. In the process of validation, it is not the test that is validated but rather the claims (interpretations and uses) about test scores that are validated. Kane's argument‐based validation framework provided the structure for reporting results where validity claims are followed by evidence to support the argument. This multivariate generalizability theory study found that the greatest source of variance was attributable to faculty raters, suggesting that portfolio assessment would benefit from two raters’ evaluating each portfolio independently. The results are generally supportive of holistic scoring, but analytical scoring deserves further research. Correlational analyses between student portfolios and traditional measures of student competence and readiness for licensure resulted in significant correlations between portfolios and National Board Dental Examination Part I (r=0.323, p<0.01) and Part II scores (r=0.268, p<0.05) and small and non‐significant correlations with grade point average and scores on the Western Regional Examining Board (WREB) exam. It is incumbent upon the users of portfolio assessment to determine if the claims and evidence arguments set forth in this study support the proposed claims for and decisions about portfolio assessment in their respective institutions.Keywords
Funding Information
- ADEAGies Foundation
This publication has 19 references indexed in Scilit:
- Validating the Interpretations and Uses of Test ScoresJournal of Educational Measurement, 2013
- Implementation of Portfolio Assessment of Student Competence in Two Dental School PopulationsJournal of Dental Education, 2012
- The use of scoring rubrics to determine clinical performance in the operating suiteNurse Education Today, 2009
- Dental Student Assessment ToolboxJournal of Dental Education, 2009
- Assessing Dental Students’ Competence: Best Practice Recommendations in the Performance Assessment Literature and Investigation of Current Practices in Predoctoral Dental EducationJournal of Dental Education, 2008
- The Dental Education EnvironmentJournal of Dental Education, 2006
- Predictive Validity of Dental Hygiene Competency Assessment Measures on One‐Shot Clinical Licensure ExaminationsJournal of Dental Education, 2005
- Standards of Validity and the Validity of Standards in Performance AsessmentEducational Measurement: Issues and Practice, 1995
- Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning.American Psychologist, 1995
- The assessment of clinical skills/competence/performanceAcademic Medicine, 1990