Using personal statements in college admissions: An investigation of gender bias and the effects of increased structure
Open Access
- 2 January 2022
- journal article
- research article
- Published by Taylor & Francis in International Journal of Testing
- Vol. 22 (1) , 5-20
Abstract
Personal statements are among the most commonly used instruments in college admissions procedures. Yet, little research on their reliability, validity, and fairness exists. The first aim of this paper was to investigate hypotheses about adverse impact and underprediction for female applicants, which could result from lower tendencies to use agentic language compared to male applicants. Second, we examined if rating personal statements in a more structured manner would increase reliability and validity. Using personal statements (250 words) from a large cohort of applicants to an undergraduate psychology program at a Dutch University, we found no evidence for adverse impact for female applicants or more agentic language use by male applicants, and no relationship between agentic language use and personal statement ratings. In contrast, we found that personal statements of female applicants were rated slightly more positively than those of males. Exploratory analyses suggest that female applicants’ better writing skills might explain this difference. A more structured approach to rating personal statements yielded higher, but still only ‘moderate’ inter-rater reliability, and virtually identical, negligible predictive validity for first year GPA and dropout.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- Employment Interview Reliability: New meta‐analytic estimates by structure and formatInternational Journal of Selection and Assessment, 2013
- Rater training revisited: An updated meta‐analytic review of frame‐of‐reference trainingJournal of Occupational and Organizational Psychology, 2011
- Fact and Fiction in Cognitive Ability Testing for Admissions and Hiring DecisionsCurrent Directions in Psychological Science, 2010
- Has the UK Clinical Aptitude Test improved medical student selection?Medical Education, 2010
- How do Educational Attainment and Gender Relate to Fluid Intelligence, Crystallized Intelligence, and Academic Skills at Ages 22-90 Years?Archives of Clinical Neuropsychology, 2009
- Answers to 20 Questions About Interrater Reliability and Interrater AgreementOrganizational Research Methods, 2007
- Noncognitive Constructs and Their Assessment in Graduate Education: A ReviewEducational Assessment, 2005
- Description and Prescription: How Gender Stereotypes Prevent Women's Ascent Up the Organizational LadderJournal of Social Issues, 2001
- Self-promotion as a risk factor for women: The costs and benefits of counterstereotypical impression management.Journal of Personality and Social Psychology, 1998
- The Personal Statement as an Indicator of Writing Skill: A Cautionary NoteEducational Assessment, 1997