Is It the Intervention or the Students? Using Linear Regression to Control for Student Characteristics in Undergraduate STEM Education Research
Authors:
Open Access
- 1 March 2014
- journal article
- Published by American Society for Cell Biology (ASCB) in CBE—Life Sciences Education
- Vol. 13 (1) , 41-48
Abstract
Although researchers in undergraduate science, technology, engineering, and mathematics education are currently using several methods to analyze learning gains from pre- and posttest data, the most commonly used approaches have significant shortcomings. Chief among these is the inability to distinguish whether differences in learning gains are due to the effect of an instructional intervention or to differences in student characteristics when students cannot be assigned to control and treatment groups at random. Using pre- and posttest scores from an introductory biology course, we illustrate how the methods currently in wide use can lead to erroneous conclusions, and how multiple linear regression offers an effective framework for distinguishing the impact of an instructional intervention from the impact of student characteristics on test score gains. In general, we recommend that researchers always use student-level regression models that control for possible differences in student ability and preparation to estimate the effect of any nonrandomized instructional intervention on student performance.Keywords
This publication has 26 references indexed in Scilit:
- Active Learning Not Associated with Student Learning in a Random Sample of College Biology CoursesCBE—Life Sciences Education, 2011
- The Effects of Problem-Based Learning Instruction on University Students’ Performance of Conceptual and Quantitative Problems in Gas ConceptsEURASIA Journal of Mathematics, Science and Technology Education, 2009
- Redesign of a Life Span Development Course Using Fink's TaxonomyTeaching of Psychology, 2008
- How Much Have They Retained? Making Unseen Concepts Seen in a Freshman Electromagnetism Course at MITJournal of Science Education and Technology, 2007
- Teaching Cell Biology to Nonscience Majors Through Forensics, or How to Design a Killer CourseCell Biology Education, 2004
- Instructional multimedia program versus standard lecture: a comparison of two methods for teaching the undergraduate orthodontic curriculumEuropean Journal of Dental Education, 2004
- The effectiveness of web-based, multimedia tutorials for teaching methods of human body composition analysis.Advances in Physiology Education, 2002
- Meta-analysis of experiments with matched groups or repeated measures designs.Psychological Methods, 1996
- Reciprocal Peer Tutoring: A Multimodal Assessment of Effectiveness with College StudentsTeaching of Psychology, 1989
- Synthesizing standardized mean‐change measuresBritish Journal of Mathematical and Statistical Psychology, 1988