Automated Text Analysis of Organic Chemistry Students’ Written Hypotheses
- 15 February 2024
- journal article
- research article
- Published by American Chemical Society (ACS) in Journal of Chemical Education
- Vol. 101 (3) , 807-818
Abstract
Generating a testable hypothesis is a necessary skill for engaging in science, requiring both general reasoning skills and specific content knowledge of the phenomenon being investigated. While many students have the reasoning skills necessary for developing testable hypotheses in a general science context, it can be challenging for students to apply these skills to content areas where the phenomena being studied are unobservable, such as problems in organic chemistry. Generating hypotheses for experiments is a skill that students are expected to apply in chemistry teaching laboratories. Students can benefit from receiving real-time feedback to support developing this ability. However, providing feedback on writing can be challenging for instructors, especially in large-enrollment courses. For this study, we explored and compared the performance of several machine learning algorithms to classify organic chemistry students’ written hypotheses for a science-general scenario and an organic chemistry-specific scenario. These models were trained to analyze students’ written hypotheses for the presence of five features: (1) a testable prediction in a given situation; (2) a predicted change in the independent variable that is related to the dependent variable; (3) scientific content addressing the research question provided in the prompt; (4) a correct and completely defined independent variable; and (5) a correct and completely defined dependent variable. This work has implications for guiding future research on the feasibility of providing instructors with information about students’ hypothesis-generating abilities that can support the delivery of real-time feedback.Keywords
This publication has 58 references indexed in Scilit:
- What Are They Thinking? Automated Analysis of Student Writing about Acid–Base Chemistry in Introductory BiologyCBE—Life Sciences Education, 2012
- Perry’s Scheme of Intellectual and Epistemological Development as a framework for describing student difficulties in learning organic chemistryChemistry Education Research and Practice, 2010
- How can Teachers Help Students Formulate Scientific Hypotheses? Some Strategies Found in Abductive Inquiry Activities of Earth ScienceInternational Journal of Science Education, 2009
- Beyond the scientific method: Model‐based inquiry as a new paradigm of preference for school science investigationsScience Education, 2008
- The Power of FeedbackReview of Educational Research, 2007
- Modelling Analysis of Students’ Processes of Generating Scientific Explanatory HypothesesInternational Journal of Science Education, 2006
- Reliability in Content Analysis.Human Communication Research, 2004
- Hypothesis testing in students: Sequences, stages, and instructional strategiesJournal of Research in Science Teaching, 1981
- An analysis of the influence of reinforcement and knowledge of criteria on the ability of students to generate hypothesesJournal of Research in Science Teaching, 1980
- Development of formal hypothesis-testing ability.Developmental Psychology, 1979