What Are They Thinking? Automated Analysis of Student Writing about Acid–Base Chemistry in Introductory Biology
Open Access
- 1 September 2012
- journal article
- Published by American Society for Cell Biology (ASCB) in CBE—Life Sciences Education
- Vol. 11 (3) , 283-293
Abstract
Students’ writing can provide better insight into their thinking than can multiple-choice questions. However, resource constraints often prevent faculty from using writing assessments in large undergraduate science courses. We investigated the use of computer software to analyze student writing and to uncover student ideas about chemistry in an introductory biology course. Students were asked to predict acid–base behavior of biological functional groups and to explain their answers. Student explanations were rated by two independent raters. Responses were also analyzed using SPSS Text Analysis for Surveys and a custom library of science-related terms and lexical categories relevant to the assessment item. These analyses revealed conceptual connections made by students, student difficulties explaining these topics, and the heterogeneity of student ideas. We validated the lexical analysis by correlating student interviews with the lexical analysis. We used discriminant analysis to create classification functions that identified seven key lexical categories that predict expert scoring (interrater reliability with experts = 0.899). This study suggests that computerized lexical analysis may be useful for automatically categorizing large numbers of student open-ended responses. Lexical analysis provides instructors unique insights into student thinking and a whole-class perspective that are difficult to obtain from multiple-choice questions or reading individual responses.Keywords
This publication has 21 references indexed in Scilit:
- Applying Computerized-Scoring Models of Written Biological Explanations across Courses and Colleges: Prospects and LimitationsCBE—Life Sciences Education, 2011
- Harnessing Technology to Improve Formative Assessment of Student Conceptions in STEM: Forging a National NetworkCBE—Life Sciences Education, 2011
- Lost in Lewis Structures: An Investigation of Student Difficulties in Developing Representational CompetenceJournal of Chemical Education, 2010
- Mapping student understanding in chemistry: The Perspectives of ChemistsScience Education, 2008
- Exploring the Characteristics and Diverse Sources of Students’ Mental Models of Acids and BasesInternational Journal of Science Education, 2007
- Assessing Students' Ability to Trace Matter in Dynamic Systems in Cell BiologyCBE—Life Sciences Education, 2006
- General Chemistry: Expanding the Learning Outcomes and Promoting Interdisciplinary Connections through the Use of a Semester-long ProjectCBE—Life Sciences Education, 2006
- Inverting Organic and Biochemistry: A Curriculum Tweak That Benefits AllJournal of Chemical Education, 2004
- Introductory Science and Mathematics Education for 21st-Century BiologistsScience, 2004
- Integrating the General Chemistry and General Biology CurriculumJournal of Chemical Education, 2000